Compare commits

...
6 Commits
Author SHA1 Message Date
Frank 7643381b9b docs(iommu): AMD PerfOpt removes the iGPU translation penalty (issue #3 research)
Records the 2026-10-18 answer to 'does a recent kernel change retire
amd_iommu=off': PerfOpt (iommu tree 2026-09-24, Linux 7.4) stops GPA->SPA
translation for the privileged iGPU while the IOMMU stays enabled for every
other device, the NPU included. Local census proves the fix is absent here
(zero perfopt symbols in System.map-7.0.0-38, archive caps at 7.0.0-38.38),
and the September 13-16% figure was translated-domain cost, not strict-flush
cost -- lazy DMA is the Ubuntu default, so iommu=pt already avoids it. The
corrected procedure therefore flips to iommu=pt now and treats PerfOpt as a
2-4% later bonus on this silicon class (Phoronix desktop arm), with the NPU
legal under identity mode because amdxdna binds its own SVA domain.
Re-keys the AGENTS.md key-lesson bullet; no host change.
2026-10-18 00:57:42 -07:00
Frank 1430f2a040 docs(log): ceiling-cut entry closes with the green 33/33 gate 2026-10-17 22:51:59 -07:00
Frank fc099d9b3c e2e: accept Plex's session-less hand-off to app.plex.tv
The full-suite re-run after the reset fix came back "1 flaky": 12-crawl reached
plex.franksdank.cloud, the guard cleared, and Plex Web — signed out in the
qa-e2e profile — handed the browser to its own hosted sign-in surface, so the
hostname never settled on the vhost (60 s poll, Expected plex.franksdank.cloud /
Received app.plex.tv). settleThroughGuard now takes a HANDOFFS landing list per
vhost; only plex.franksdank.cloud gets one, every other host still has to land on
its own vhost, and the guard assertions (Anubis cleared, no authentik uidField,
status < 500) are unchanged. Worklog and E2E gotcha 6 carry both symptoms: the
ERR_ABORTED reset and the hand-off.
2026-10-17 22:49:35 -07:00
Frank d2587e1d29 e2e: crawl reset survives app navigation; dragoon docs carry the cut ledger
Two faces of the same verification round.

12-crawl returned to the homepage between links with a bare page.goto, and Plex
Web (signed out, so it renders its own sign-in iframe for its Plex-account flow)
keeps redirecting the top frame: Chromium cancelled our goto with
net::ERR_ABORTED, outside the try, so the whole crawl died — once as a 420 s
timeout in the deploy gate, once at 5.4 s on the same link. backHome() tolerates
that one error class and re-issues; any other error still propagates, and the
per-app verdict (guard settled, hostname, no authentik uidField, status < 500)
is unchanged. Plex was healthy throughout (container up 2 days, 32400 answering
HTTPS 401 / plain-HTTP empty reply), so this was app behaviour, not a fault —
recorded as gotcha 6 on the E2E page.

The 200,704 boot printed 5.5 GiB reserved and 74.1 GiB in all (not the ~5.4/~74
my arithmetic predicted), so every prose figure is re-keyed to the ledger, and
the runbook gains the second production-boot section: marker revision, pool
reservation, shape line, prompt-cache entry size, and the refusal contract at the
new ceiling (400 in 0.86 s naming 200704 and the 96,405-token overage, before any
prefill). Cost stated (two maximum-length conversations never coexist; the 180k
compaction target is unreachable inside a 155,648 window) and purchase stated
(1.7 GiB of page cache, 9.1 GiB against the 2026-10-06 shape).
2026-10-17 22:45:34 -07:00
Frank 7c4dbebe24 frank-dragoon: lower the context to 200704 (ctx and pool together)
Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and
HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung
frank-zealot already runs). The checkpoint's native window is 262,144 and the
engine serves anything below it, so this is a RAM decision, not a capability
limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured
14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table
loses -- the freed positions become page cache for the 47.68 GiB table that
long prompts read. Engine-side total ~74 GiB instead of 75.8.

The 400-zone law is invariant to the value, so the harness moves WITH the
ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping
the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs
the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est
(~136k measured); the 180k target of the 262,144 era is unreachable because
the window itself is 155,648. maxTokens 45056 and the coupled
cap/timeout 65536/7200 are untouched.

Cost stated plainly: with pool == ctx, ONE maximum-length request fills the
pool, so two long conversations never coexist (two of <=100k do). Acceptable
because frank-dragoon carries only the slow role and the security-reviewer
subagent, not the interactive workhorse.

Seam re-keyed: the fit-law test now proves the invariant at the new pair, and
the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
2026-10-17 22:29:22 -07:00
Frank 860386d283 docs(frank-dragoon): production boot ledger, worklog, retired-era ctx fix
Deploy verification for fcd7630 (22:20): healthy in ~12 s, e2e 33/33, and the
boot ledger settles the shape — 62.9 GiB weights locked (0.8 vision) + 7.2 GiB
KV in ONE 262,144-position pool (6,656 MiB, two slots) + 5.7 GiB working =
75.8 GiB in all, n-gram 47.68 GiB read from disk, prompt cache ON (12 entries),
MTP depth-2, no clamp line. Live probes: 27*59 -> 1593, tool call, red PNG ->
Red. The two KV figures are reconciled (engine preflight FORMULA 7.4 GiB
includes block scratch + per-slot O(1) state; the RESERVATION is 7.2).

Also corrects two stale claims: the retired swift15 rows named 200,192 ctx
(raised to 262,144 on 2026-10-07 inside that same era), and the OpenWebUI
frank-dragoon row's description is now re-keyed in the live DB (backup
webui.db.bak-20261017b).
2026-10-17 22:23:59 -07:00
12 changed files with 467 additions and 134 deletions

No files matched your search

+40 -29
View File
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=262144` pool** (the card's recommendation: the ledger's 7.2 GiB where the old 524,288 pool measured 14.6 — per-request ceiling unchanged at 262,144, two maximum-length conversations no longer coexist, and the freed RAM is page cache for the n-gram table), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=200704` pool at `HALOGEN_CTX=200704`** (lowered from the card's 262,144 the same evening — 5.5 GiB reserved where 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; pool == ctx so ONE maximum-length request fills it, and the freed RAM is page cache for the n-gram table), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). | | Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
Retired endpoints (`frank-npu`, and every prior runtime) are in Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -380,25 +380,29 @@ endpoint).
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
policy disables the MTP head for whole requests after low acceptance and policy disables the MTP head for whole requests after low acceptance and
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
for it (accepted). `HALOGEN_CTX=262144` is the PER-SLOT hard ceiling (400 for it (accepted). `HALOGEN_CTX=200704` is the PER-REQUEST hard ceiling (400
naming the limit, never truncation) — RAISED from 200192 on 2026-10-07 naming the limit, never truncation) — the checkpoint's native window is
under the 400-zone law: the harness clamps max_tokens to 262,144 and was served until 2026-10-17, when Frank lowered it for RAM. The
**400-zone law is invariant to the value**: the harness clamps max_tokens to
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts `min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388, the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
request 43,158 vs room 42,804), so a harness window == engine ceiling request 43,158 vs room 42,804), so a harness window == engine ceiling
made every request over ~146k est a guaranteed 400. `models.yml` makes every long request a guaranteed 400. `models.yml`
`contextWindow: 217088` leaves the 45k gap that absorbs the estimate `contextWindow: 155648` (= ceiling − 45056) keeps that gap (worst-case
error (worst-case request ≈ 219k ≤ 262144); compaction fires at 85 % of request ≈ 156.2k ≤ 200,704); compaction fires at 85 % of the harness window
the harness window = 184,524 est ≈ 190k measured (Frank's 180k target). = 132,301 est ≈ 136k measured (the 180k target died with the 262,144 window).
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter" `HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
law does NOT carry over; batching and speculation coexist (the card's law does NOT carry over; batching and speculation coexist (the card's
quickstart ships 2 slots with the head), so 2 is concurrent capacity. quickstart ships 2 slots with the head), so 2 is concurrent capacity.
`HALOGEN_KV_POOL_POSITIONS=262144` (the card's recommendation and the memory `HALOGEN_KV_POOL_POSITIONS=200704` — **lowered from the card's 262,144 the same
saving of the 2026-10-17 swap; ~28 KiB/position ⇒ 7.2 GiB) is **ONE pool evening** (Frank's call: every position costs KV RAM and nothing in the fleet
behind both slots** — ledger `2 slots over ONE 262144-position KV pool, each asks for the biggest window) — is **ONE pool behind both slots**, ledger form
request up to 262144` — so the per-request ceiling is untouched while two `2 slots over ONE <pool>-position KV pool, each request up to HALOGEN_CTX`, and
MAXIMUM-length conversations no longer coexist (the old 524,288 = 2 × ctx at pool == ctx a single MAXIMUM-length request fills it: two maximum-length
spent the measured 14.6 GiB and allowed it). conversations never coexist (two of ≤100k do). ~28 KiB/position ⇒ the ledger
reserves "about 5.5 GiB", where 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6; the
engine's preflight FORMULA bills slightly more than the reservation (block
scratch + ~115 MiB/slot of O(1) state) — quote the reservation.
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` + harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table). `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
@@ -415,16 +419,17 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
`image_url` content parts only; ≥256 px probe images or the model answers `image_url` content parts only; ≥256 px probe images or the model answers
"video". "video".
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger): **Memory ledger** (121.2 GiB unified; the referee is the boot ledger — the
weights ~62.9 (trunk 62.11 + vision 0.84) + KV 7.2 + arena ~8 200,704 boot printed exactly): weights 62.9 (trunk 62.11 + vision 0.84) + KV
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ 78 GiB engine-side + 5.5 + working 5.7 (`3.8 + 8.4 × MAX_TOK/16384`) = **74.1 GiB in all**, the
`HALOGEN_HOST_RESERVE_GIB=20` → ~25 GiB margin over baseline co-tenancy. ~1.0 GiB prompt cache booked separately by the ledger, +
The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint `HALOGEN_HOST_RESERVE_GIB=20` → ~30 GiB
(ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why the pool margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd page
cut is a double saving: **RAM the pool takes is RAM that table loses**. cache beside the checkpoint (ledger: `mapped 51.2 GB, NOT pinned
`HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the v2 loader, so disk-backing (Pin::None)`) — which is why every pool cut is a double saving: **RAM the pool
the trunk is not a lever on this format. READ the boot ledger and re-size takes is RAM that table loses**. `HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the
if the engine prints a clamp. v2 loader, so disk-backing the trunk is not a lever on this format. READ the
boot ledger and re-size if the engine prints a clamp.
Fetch: `scripts/download-frank-dragoon-v2abl.sh` into Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS `/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
@@ -1103,9 +1108,15 @@ linked runbooks.
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its - SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
ports off the gluetun service. ports off the gluetun service.
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon, - `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind 2026-09-11) retires the XDNA2 NPU: the driver's SVA/PASID bind needs an
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). That trade is on an
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`. expiring path: **AMD PerfOpt** (iommu tree 2026-09-24, Linux 7.4) stops
iGPU GPA→SPA translation while the IOMMU stays enabled for every other
device, so the next boot-line window is `iommu=pt` with the NPU usable —
measured only 2–4 % residual on a Strix Halo desktop (18–23 % on Strix
Point laptops). Not installable here yet: Ubuntu 26.04 caps at
`7.0.0-38`. Full research in
`serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers - Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
only the CURRENT instance; history across recreations lives in only the CURRENT instance; history across recreations lives in
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo `journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
@@ -1156,7 +1167,7 @@ Active:
| Service | Runbook | | Service | Runbook |
|---------|---------| |---------|---------|
| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 262,144 pool, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` | | frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 200,704 pool at ctx 200704, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` | | frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` | | frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` | | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
@@ -1169,7 +1180,7 @@ Retired (rollback targets and history):
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` | | radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` | | llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` | | llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, 2 slots × 200,192 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` | | halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, TWO slots at ctx 262,144 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` | | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` | | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
+3 -3
View File
@@ -1097,9 +1097,9 @@ deploy_anubis() {
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next — # conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next —
# qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident) # qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident)
# + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode # + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode
# (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 262,144-position KV pool, image # (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 200,704-position KV pool at the
# input — on the Strix Halo iGPU. It stays a ROOT unit because the serve # matching ctx 200704 ceiling, image input — on the Strix Halo iGPU. It stays a
# script drives `podman run` directly (the frank-archon pattern): # ROOT unit because the serve script drives `podman run` directly (the frank-archon pattern):
# relink_unit points /etc/systemd/system/frank-dragoon.service at the repo # relink_unit points /etc/systemd/system/frank-dragoon.service at the repo
# file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity # file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity
# is fleet parity. 2026-10-17 cutover from the Swift1.5-finetune shape — # is fleet parity. 2026-10-17 cutover from the Swift1.5-finetune shape —
+50 -42
View File
@@ -20,6 +20,10 @@
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the # thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
# base thinks ~1.9x longer per answer. # base thinks ~1.9x longer per answer.
# #
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered both
# to 200,704 (RAM + the harness's compaction budget); the rates above are the
# ceiling-262,144 record and the production ledger is the 200,704 one.
#
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own # WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2 # card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
# .hgn loads ONLY in the flash-server line (the old halogen engine is for # .hgn loads ONLY in the flash-server line (the old halogen engine is for
@@ -29,50 +33,54 @@
# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT # the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint — # return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
# red-PNG probe verified on the card; the 27B engine ignored its encoder), # red-PNG probe verified on the card; the 27B engine ignored its encoder),
# tool calls verified, and the native 262,144 context. # tool calls verified, and the native 262,144 context — served at 200,704 by
# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
# capability limit).
# #
# SHAPE — the knobs that are NOT free, with the arithmetic: # SHAPE — the knobs that are NOT free, with the arithmetic:
# HALOGEN_CTX=262144 is the PER-SLOT window and the per-conversation hard # HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
# ceiling — a prompt over it is a 400 naming the limit, never a silent # hard limit — a prompt over it is a 400 naming the limit, never a silent
# truncation. RAISED from 200192 on 2026-10-07: the harness clamps # truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
# max_tokens to contextWindow − est − 4096, its chars/4 est UNDER-counts # checkpoint's native window is 262,144 and the engine serves anything
# the rendered prompt by ~3 % on code/agent traffic, and compaction only # below it): every position costs KV pool RAM and shrinks the prompt-cache
# fires at 85 % — so with window == engine ceiling, every request over # entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
# ~146k est (65-85 % of the window) was a guaranteed 400. Measured # to the value — it says the harness window must sit ~45k UNDER the engine
# 2026-10-07: est 152,938, server 157,388, request 43,158 vs room 42,804 # ceiling, because the harness clamps max_tokens to contextWindow − est −
# — over by 354, margin 4,096 < the 4,450 under-count. # 4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
# Harness contract (models.yml): contextWindow 217088, maxTokens 45056. # code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
# Compaction fires at 85 % = 184,524 est ≈ 190k measured (Frank's 180k # at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 % # Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and # the gap that absorbs the estimate error), maxTokens 45056.
# engine ceiling absorbs the estimate error with ~7x margin (robust to a # Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
# 10 % under-count). Compaction rides on frank-zealot: input ≈ # of the 262,144 era is unreachable at this ceiling: the window itself is
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot. # 155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
# HALOGEN_KV_POOL_POSITIONS=262144 is the card's recommended pool and the # est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
# memory saving of this swap. One budget behind HALOGEN_KV_SLOTS=2 (boot # ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
# ledger: "2 slots over ONE 262144-position KV pool, each request up to # keepRecent(20k) ≈ 112k, inside its 200,704 slot.
# 262144"), so the per-request ceiling stays HALOGEN_CTX while two # HALOGEN_KV_POOL_POSITIONS=200704 — ONE pool behind HALOGEN_KV_SLOTS=2
# MAXIMUM-length conversations can no longer coexist — two moderate ones # (ledger form: "2 slots over ONE <pool>-position KV pool, each request up to
# can. ~28 KiB/position (FLAGS.md) => 7.2 GiB, and the ledger said exactly # HALOGEN_CTX"). With pool == ctx a single MAXIMUM-length request fills the
# that: "reserving the KV pool, 262144 positions (about 7.2 GiB)". The # whole pool. ~28 KiB/position (FLAGS.md) ⇒ ~5.4 GiB, where 262144 cost 7.2
# 2026-10-06 shape's 524,288 cost 14.6 GiB; FLAGS.md law — "RAM the pool # and the 2026-10-06 shape's 524,288 measured 14.6. The engine's own preflight
# takes is RAM that table loses" — makes the 7.4 GiB handed back page # formula bills a little more than the reservation because it adds block
# cache for the 47.7 GiB n-gram table, which is what long prompts read. # scratch and ~115 MiB/slot of O(1) state — quote the RESERVATION for the RAM
# ledger. FLAGS.md law: "RAM the pool takes is RAM that table loses", so every
# position cut is page cache for the 47.68 GiB n-gram table long prompts read.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP # HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation # drafter is NOT slot-locked on this engine (batching and speculation
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own # coexist — DFlash-era slots>1 law does NOT carry over; the card's own
# quickstart ships 2 slots WITH the head), so the second slot is # quickstart ships 2 slots WITH the head), so the second slot is
# concurrent capacity, not a speculation tax. # concurrent capacity, not a speculation tax.
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is # Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the # mmap'd page cache beside the checkpoint, not a pinned charge — the boot
# card's own quickstart arithmetic does not bill it, and the boot ledger # ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
# agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"): # weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~5.4 +
# weights ~62 GiB resident + KV 7.2 + arena (HALOGEN_MAX_TOK=8192 => # working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 74 GiB engine-side, with the
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2 # prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked
# = ~79 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host # separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ) # honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
# ~10-15. ~25 GiB margin. The boot ledger is the referee — read it # ~10-15. ~30 GiB margin. The boot ledger is the referee — read it after
# after deploy and re-size if the engine prints a clamp. # deploy and re-size if the engine prints a clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy # HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
# turns the MTP head off for whole requests after low acceptance and # turns the MTP head off for whole requests after low acceptance and
# HALVES decode as a result. Frank accepted the head always-on trade # HALVES decode as a result. Frank accepted the head always-on trade
@@ -99,11 +107,11 @@
# so the models.yml fold (high->xhigh) still applies. # so the models.yml fold (high->xhigh) still applies.
# * Vision: OpenAI image_url content parts only (the fleet's standard # * Vision: OpenAI image_url content parts only (the fleet's standard
# contract). >=256 px probe images or the model answers "video". # contract). >=256 px probe images or the model answers "video".
# * TWO maximum-length conversations no longer share the endpoint: the KV # * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
# pool is one 262,144-position budget behind 2 slots. The PER-REQUEST # budget behind 2 slots — exactly one maximum-length request fills it. Two
# ceiling is unchanged (HALOGEN_CTX), so the harness contract below # conversations coexist only while they stay under ~100k each. The
# (contextWindow 217088) does not move; only concurrency at full depth # PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
# does, and this endpoint carries the slow + security-reviewer roles. # contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
# * response_format json_object/json_schema unsupported in this line # * response_format json_object/json_schema unsupported in this line
# (no constrained decoding) — unchanged from the 27B engine. # (no constrained decoding) — unchanged from the 27B engine.
# * The engine's token protocol has no auth and binds loopback INSIDE # * The engine's token protocol has no auth and binds loopback INSIDE
@@ -165,8 +173,8 @@ FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_conf
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs # scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
# model_ids for the :7770 backend MUST name this value. # model_ids for the :7770 backend MUST name this value.
HALOGEN_MODEL_ID=frank-dragoon HALOGEN_MODEL_ID=frank-dragoon
HALOGEN_CTX=262144 HALOGEN_CTX=200704
HALOGEN_KV_POOL_POSITIONS=262144 HALOGEN_KV_POOL_POSITIONS=200704
HALOGEN_KV_SLOTS=2 HALOGEN_KV_SLOTS=2
HALOGEN_HOST_RESERVE_GIB=20 HALOGEN_HOST_RESERVE_GIB=20
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD); # Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 262144-position pool, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU) Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 200704-position pool at ctx 200704, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -82,3 +82,14 @@ real IdP credentials.
5. **Solve-once cookie.** Since Anubis v1.27.0 + `COOKIE_DOMAIN` (see 5. **Solve-once cookie.** Since Anubis v1.27.0 + `COOKIE_DOMAIN` (see
[[Anubis]]), one PoW per fresh context covers every guarded vhost — crawl [[Anubis]]), one PoW per fresh context covers every guarded vhost — crawl
and SSO specs complete in ~0.5 s per vhost instead of a PoW each. and SSO specs complete in ~0.5 s per vhost instead of a PoW each.
6. **Apps navigate too — and may leave.** `12-crawl` returns to the homepage
between links, and Plex Web (signed out in the `qa-e2e` profile) runs its own
sign-in flow: it redirects the top frame, so Chromium cancelled our
`goto("/")` with `net::ERR_ABORTED` (once as a 420 s timeout, once at 5 s on
the same link), and the same redirect then lands the browser on
`app.plex.tv` — Plex's hosted surface for a session-less client — which broke
the `hostname == target` law. Two narrow answers: `backHome()` re-issues the
navigation on that one error class, and the landing check takes an explicit
hand-off list (`HANDOFFS`) instead of a single host. The guard contract is
untouched: challenge cleared, no authentik `uidField`, status < 500, and every
host NOT in the list still has to land on its own vhost.
@@ -89,6 +89,70 @@ with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
endpoint is wanted back. The IOMMU change is independent of the halogen endpoint is wanted back. The IOMMU change is independent of the halogen
deployment — reverting one does not require reverting the other. deployment — reverting one does not require reverting the other.
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
Asked and answered — **the decision changes, the host does not.** Nothing was
changed on Dankclaw today; this section records why the 13–16 % figure is on an
expiring path.
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation
for the *privileged integrated GPU* — only the IR/IW permission bits are
enforced — while the IOMMU stays fully enabled for every other device, the NPU
included. Driver: `drivers/iommu/amd/{iommu.c,amd_iommu_types.h}`. The only
consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and
the identity domain**, and the shared per-IOMMU bit is refcounted so one
device's teardown cannot clear it under a peer.
**Status.** v2 merged through the iommu tree 2026-09-24, present in
`linux-next-20261016`, queued for **Linux 7.4** (merge window late October 2026,
stable ~end of year, likely LTS candidate). NOT available here: `/boot/
System.map-7.0.0-38-generic` contains zero `perfopt` symbols, and Ubuntu 26.04's
archives cap at `7.0.0-38.38` for both `linux-image-generic` and
`linux-generic-hwe-26.04-edge` — reaching 7.4 means a mainline PPA or a local
build. A 2026-09-28 follow-up removes the `amdgpu.iommu_perfopt` knob and makes
PerfOpt compulsory.
**Measured.** Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel,
`amdgpu.iommu_perfopt=0` vs default): **18–23 %** on Strix Point / Krackan
laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our
silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02).
**The September number was never strict-flush cost.** Ubuntu ships
`CONFIG_IOMMU_DEFAULT_DMA_LAZY=y` (STRICT not set), so IOTLB invalidation was
already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce.
What we measured was **translated-domain** cost — and `iommu=pt` avoids that
today, with or without PerfOpt. The March attempt to make passthrough the AMD
default was rejected outright, so `iommu=pt` / `iommu.passthrough=1` stays the
sanctioned route.
**The NPU is compatible with identity mode.** `amdxdna` binds through
`iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`),
and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that
is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA
failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7
disables SVA outright**; skip that range.
**Container access needs no reboot to settle:** `/dev/accel/accel0` already
exists as `crw-rw---- root:render 261,0` and `dank` is in `render`, so a quadlet
wants `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same
pattern ComfyUI uses for `/dev/kfd`.
**Revised procedure for the next boot-line window** (still a maintenance window,
still an explicit OK per run):
1. `iommu=pt` in place of `amd_iommu=off` — NPU usable, translated cost gone,
residual 2–4 % on this silicon class.
2. Re-measure prefill against the **current** payload. The 13–16 % belongs to the
2026-09-11 shape; the 2026-10-17 native-v2 boot runs 1,513–1,604 t/s cold
prefill, so the old baseline is not comparable.
3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity
domain. Confirm with `dmesg | grep -i perfopt`; force off with
`amdgpu.iommu_perfopt=0` while the knob still exists.
## Related ## Related
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause - [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
@@ -98,3 +162,9 @@ deployment — reverting one does not require reverting the other.
## Citations ## Citations
- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8 - [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
- [iommu/amd: Add PerfOpt IOMMU performance optimization support (v2)](https://lists.freedesktop.org/archives/amd-gfx/2026-September/152501.html) — series 0/2 at [/153378.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153378.html), merged via the iommu tree 2026-09-24 ([/153844.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153844.html))
- [Compulsory-PerfOpt follow-up (removes `amdgpu.iommu_perfopt`)](https://patchew.org/linux/20260928045050.955165-1-superm1@kernel.org/)
- [Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4](https://www.phoronix.com/review/amd-perfopt) — Strix Halo desktop arm (2–4 %), `amdgpu.iommu_perfopt=0` A/B; news write-up [here](https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4)
- [Robin Murphy rejecting the AMD passthrough-default patch](https://lkml.org/lkml/2026/3/26/924) — `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned path
- [Debian wiki: XDNA](https://wiki.debian.org/XDNA) — driver + firmware upstreamed in 6.14; **amdxdna broken on 6.18–6.18.7** by an IOMMU regression that disables SVA ([report](https://lore.kernel.org/regressions/870872aa-28e9-412a-bac6-8020bf560e4f@amd.com/))
- [amd/xdna-driver#704](https://github.com/amd/xdna-driver/issues/704) — the `err=-19` bind failure when the IOMMU is disabled
@@ -66,9 +66,11 @@ ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b8680670
## Shape — the card's settings, and the three deliberate deviations ## Shape — the card's settings, and the three deliberate deviations
`HALOGEN_CTX=262144` (per-request ceiling), `HALOGEN_KV_SLOTS=2`, `HALOGEN_CTX=200704` (per-request ceiling — the card's 262,144 lowered the
**`HALOGEN_KV_POOL_POSITIONS=262144`** (the card's recommendation, and the same evening, see below), `HALOGEN_KV_SLOTS=2`,
memory saving), `HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`, **`HALOGEN_KV_POOL_POSITIONS=200704`** (one pool behind both slots; the card
recommends it equal to ctx, and that equality is where the memory is),
`HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair), `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`, sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
`HALOGEN_HOST_RESERVE_GIB=20`. `HALOGEN_HOST_RESERVE_GIB=20`.
@@ -81,24 +83,30 @@ Deviations from the card, each on purpose:
(the adaptive default disables the drafter per request and halves decode). (the adaptive default disables the drafter per request and halves decode).
- the sampler values are peonist's engine recipe; Quat3rnion publishes none. - the sampler values are peonist's engine recipe; Quat3rnion publishes none.
**The pool is one budget, not a partition.** Boot ledger: `listening on **The pool is one budget, not a partition.** Boot ledger form: `listening on
127.0.0.1:8730, 2 slots over ONE 262144-position KV pool, each request up to 127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to
262144`. The per-request ceiling is unchanged, so the harness contract <HALOGEN_CTX>`. Both numbers were lowered to **200,704** the same evening
(`contextWindow: 217088`) does not move — what changed is that **two (the card ships 262,144; Frank took the RAM — nothing else in the fleet asks
maximum-length conversations can no longer share the endpoint**. That is for the bigger window), so `models.yml` `contextWindow` moved to **155,648**
acceptable here because frank-dragoon carries the `slow` (medium-thinking) = ceiling − 45k with it. With pool == ctx, one maximum-length request fills
role and the `security-reviewer` subagent, not the interactive workhorse the pool: **two long conversations can no longer share the endpoint** (two of
(that is frank-archon). ≤100k can). That is acceptable here because frank-dragoon carries the `slow`
(medium-thinking) role and the `security-reviewer` subagent, not the
interactive workhorse (that is frank-archon).
**Memory ledger** (121.2 GiB unified), from the boot ledger of the live **Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
candidate: `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk, printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about `0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in 7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …`. The n-gram RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in
table rides as page cache — `mapped 51.2 GB, NOT pinned (Pin::None)` — which all`. The cut boot printed the same shape at the new size: `reserving the KV
is exactly why cutting the pool is a *double* saving. FLAGS.md law: **RAM the pool, 200704 positions (about 5.5 GiB)` and `… 5.5 GiB of KV pool, 5.7 GiB of
pool takes is RAM that table loses**; the 7.4 GiB the smaller pool hands back working memory, 74.1 GiB in all`.
is page cache for the 47.68 GiB table that long prompts read. The n-gram table rides as page cache — `mapped 51.2 GB,
NOT pinned (Pin::None)` — which is exactly why every pool cut is a *double*
saving. FLAGS.md law: **RAM the pool takes is RAM that table loses**; the ~9
GiB the two cuts together handed back is page cache for the 47.68 GiB table
that long prompts read.
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk `HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
stays pinned. Disk-backing the weights is not a lever on this format. stays pinned. Disk-backing the weights is not a lever on this format.
@@ -185,3 +193,79 @@ mandatory); unknown sampling params are a 400; the prompt cache is on by
default, so a warm 100k follow-up costs seconds. See default, so a warm 100k follow-up costs seconds. See
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that [[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
carries verbatim, and [[deployment]] for the deploy/health gate. carries verbatim, and [[deployment]] for the deploy/health gate.
## Production boot — the deploy ledger (2026-10-17 22:20, `fcd7630`)
`./scripts/deploy.sh dankclaw --service frank-dragoon`: healthy after ~12 s,
deploy marker `fcd7630`, e2e gate 33/33. The trial drop-in went to
`~/.AGENTTRASH/20261017-v2abl-trial/` FIRST, so this boot is governed by the
repo env — `systemctl show -p DropInPaths` is empty.
Ledger lines that settle the shape (engine stdout, not the journal):
|Line|Figure|
|---|---|
|preflight|`MemAvailable now 105.4 GiB` against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache|
|weights|`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)`|
|KV pool|`reserving the KV pool, 262144 positions (about 7.2 GiB)`; `the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions`; slots `1 -> 2 POOLED, 111.0 MiB each of O(1) state`|
|working memory|5.7 GiB at `HALOGEN_MAX_TOK 8192`|
|total|`75.8 GiB in all` — ~25 GiB of the 121.2 GiB box left for co-tenants|
|n-gram|`47.68 GiB from /models/qwen38-flash-next-ngram.hgn`, `checkpoint: mapped 51.2 GB, NOT pinned (Pin::None)` — disk-backed exactly as designed|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, `HALOGEN_CACHE_INPLACE=1` (the KV rows are never copied)|
|spec decode|`MTP depth-2 selectable (drafter 1, speculates while alone)`|
|clamps|none — the requested pool is the delivered pool|
Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB
(~28 KiB/position **including** block scratch and ~115 MiB/slot of O(1) state)
while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the
RAM ledger and the formula when comparing pool sizes.
Live probes on the deployed endpoint: `27*59` → `1593` with a warm thinking
channel, tool call `{"city": "Seattle"}`, 512 px solid-red PNG → **Red** (320
prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s
on the FIRST request after boot, i.e. the warmup artifact; the trial's settled
figure is 50.0. OpenWebUI's `frank-dragoon` row description re-keyed to the
native payload (backup `webui.db.bak-20261017b`; no container restart forced —
forcing it logs every family session out for one cosmetic string).
## Ceiling lowered to 200,704 — the second production boot (`7c4dbeb`)
The same evening, after the ledger above was read: `HALOGEN_CTX` and
`HALOGEN_KV_POOL_POSITIONS` 262144 → **200704** (49 × 4096, the per-slot rung
the llama.cpp endpoint frank-zealot already runs). The checkpoint's native
window is 262,144 and the engine serves anything below it, so this is a RAM
decision, not a capability limit. Nothing in the fleet asks for the bigger
window; the harness moved WITH the ceiling (400-zone law), `models.yml`
`contextWindow: 155648`.
|Line|Figure|
|---|---|
|marker|`revision=7c4dbeb…`, `status=deployed`; unit `active`, `/health` 200|
|KV pool|`reserving the KV pool, 200704 positions (about 5.5 GiB)`|
|shape|`2 slots over ONE 200704-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), max_tok 8192 … MTP depth-2 selectable (drafter 1…)`|
|working|5.7 GiB (unchanged — it scales with `HALOGEN_MAX_TOK`, not the window)|
|total|`62.9 GiB of weights locked … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all`|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, **111 MiB each at ctx 200704**|
|clamps|none — requested pool == delivered pool|
The refusal contract at the new ceiling, asked for explicitly (the 1.29 MB
wikitext corpus through `/v1/chat/completions`):
```
HTTP 400 {"error":{"message":"the prompt does not fit: it is 297109 tokens and
the context is 200704, 96405 tokens over. Shorten the prompt, or raise the
context (HALOGEN_CTX, and HALOGEN_ROPE_YARN for more than the native 262144).",
"type":"invalid_request_error"}}
```
Refused in 0.86 s — the preflight rejects before any prefill, so probing the
ceiling costs the endpoint nothing. OpenWebUI's `frank-dragoon` row description
re-keyed to "ONE 200,704-position KV pool at ctx 200,704" (still no container
restart; `webui.db.bak-20261017b` holds the pre-cutover row).
**What the cut cost:** with pool == ctx one maximum-length request fills the
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
is unreachable because the window itself is smaller. **What it bought:** 1.7
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
+7 -6
View File
@@ -19,7 +19,7 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
digest serving the NATIVE v2 conversion of the abliterated digest serving the NATIVE v2 conversion of the abliterated
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine, Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
262,144-position KV pool, vision ON)** and 200,704-position KV pool at ctx 200704, vision ON)** and
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml **llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48 Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled, Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
@@ -54,7 +54,7 @@ gfx1151 iGPU 114 GiB GTT]
E[flash-server container gfx1151, digest-pinned image E[flash-server container gfx1151, digest-pinned image
frank-dragoon, port 7770] frank-dragoon, port 7770]
F[flash-server v2 native abliterated .hgn + ngram table + vision tower F[flash-server v2 native abliterated .hgn + ngram table + vision tower
ONE 262144-position KV pool, 2 slots, MTP always-on, image input] ONE 200704-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
G[llama.cpp PrismML fork CUDA build G[llama.cpp PrismML fork CUDA build
frank-zealot, port 7771] frank-zealot, port 7771]
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
@@ -87,9 +87,10 @@ frank-zealot, port 7771]
**hardlinked** beside it — the card publishes only the main file, and the **hardlinked** beside it — the card publishes only the main file, and the
engine's own tools certified the stock companions (`niah` 18/18, `ppl` engine's own tools certified the stock companions (`niah` 18/18, `ppl`
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus). 3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
Card-recommended `HALOGEN_KV_POOL_POSITIONS=262144`: **one pool behind two `HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=200704`: **one pool
slots** — 7.2 GiB where the old shape spent 14.6, the per-request ceiling behind two slots** (the card ships 262,144/262,144 = 7.2 GiB; Frank took the
unchanged at 262,144, and the freed RAM becomes page cache for the n-gram RAM the same evening — 5.5 GiB reserved, one maximum-length request filling
the pool), and the freed RAM becomes page cache for the n-gram
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5 65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
@@ -98,7 +99,7 @@ frank-zealot, port 7771]
stay on disk stay on disk
- [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17 - [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17
(rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned (rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned
v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots × 200,192 ctx over v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots at ctx 262,144 over
a 524,288-position pool; owns the 400-zone law (the harness chars/4 a 524,288-position pool; owns the 400-zone law (the harness chars/4
under-count against the engine ceiling), the MTP-always-on law, and the under-count against the engine ceiling), the MTP-always-on law, and the
sampler / prompt-cache / vision contract the native era carries verbatim sampler / prompt-cache / vision contract the native era carries verbatim
+4 -3
View File
@@ -313,9 +313,10 @@ The LLM inference fleet on the two hosts. Full index:
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook - [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2 since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
abliterated** conversion of orcarouter's pinned BF16 abliterated** conversion of orcarouter's pinned BF16
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) with the card's (`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx/pool **200,704**
memory-saving `HALOGEN_KV_POOL_POSITIONS=262144` (7.2 GiB, one pool behind (`HALOGEN_KV_POOL_POSITIONS=200704`, ~5.4 GiB — one pool behind both slots,
both slots) and the stock peonist n-gram/vision companions **hardlinked** lowered the same evening from the card's 262,144) and the stock peonist
n-gram/vision companions **hardlinked**
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
always-on, vision ON, no server-side sampler change always-on, vision ON, no server-side sampler change
- [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17 - [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17
+111
View File
@@ -1896,3 +1896,114 @@ every restart since the cutover.
hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook
header corrected (was still "LIVE", never updated at retirement) and its header corrected (was still "LIVE", never updated at retirement) and its
Rollback table rebuilt. Rollback table rebuilt.
## 2026-10-17 — frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool
- Ask: move frank-dragoon to `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
with the card's recommended settings, save memory, keep the n-gram table
disk-backed. Census first: the endpoint carries only the `slow` +
`security-reviewer` roles (the interactive workhorse is frank-archon), the
table was ALREADY disk-backed (`mapped 51.2 GB, NOT pinned (Pin::None)`), and
`HALOGEN_FLASH_PIN_TRUNK=0` is rejected by the v2 loader — so the only real
lever was the KV pool: 524,288 → the card's 262,144.
- **Filename trap caught before the fetch:** this repo name carried two
different artifacts. The writer-transplant era's file measured
66,687,678,464 B; the native 2026-10-11 conversion is 66,687,678,432 B —
32 bytes apart. Byte-exact gate + LFS sha256 `b5bd925f…` in
`scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the peonist
n-gram/vision/tokenizer trio (the release publishes only the main file; a
hardlink is a regular file, so the no-symlink serve gate passes without a
47.7 GiB copy).
- **Companion risk adjudicated by the engine, not by argument.** The card does
NOT establish the stock peonist companions as matched to a native conversion
and publishes no source-matched pair, so the A/B ran the checkpoint tools on a
stopped service under one knob set (`scripts/census-frank-dragoon.sh`):
`verify` OK, **NIAH 18/18 vs 17/18**, PPL 3.968 vs 3.877 (+2.3 % — the
finetune's edge is gone), KL vs the swift15 logits mean 0.155 / median 0.057
with top-1 85.1 % = two sound models disagreeing on content, not a corrupt
forward pass. Behavior battery (`scripts/battery-frank-dragoon.py`) on both
boots: refusals 0/5 both, tools/vision identical, needles hit at both depths,
decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 (+12.6 %), thinking
tokens 1,476 vs 784 (the finetune compressed reasoning ×1.9).
- Boot window run through a transient `FRANK_DRAGOON_ENV_FILE` drop-in — never
a scratch port (one iGPU, one server) — with the stop+mask law and a
trap-guarded restore (`scripts/trial-frank-dragoon.sh`). Masking fails on a
relinked unit file, as documented; stop alone was enough.
- Deployed at 22:20 (`fcd7630`, `--service frank-dragoon`): healthy in ~12 s,
e2e 33/33. Production ledger: 62.9 GiB weights locked (0.8 vision) + 7.2 GiB
KV (ONE 262,144-position pool, 6,656 MiB, two slots) + 5.7 GiB working =
**75.8 GiB in all**, no clamp line, n-gram 47.68 GiB read from disk, prompt
cache ON with 12 entries, MTP depth-2. The trial drop-in was moved to
`~/.AGENTTRASH/20261017-v2abl-trial/` before the deploy so the repo env
governs (`DropInPaths=` empty). Live probes: `27*59` → 1593, tool call
`{"city": "Seattle"}`, 512 px red PNG → Red (320 prompt tokens).
- Files moved: env (payload dir/checkpoint/bytes/sha256/pool + the measured
A/B in the header), serve script + deployer (paths, fetch hints), unit
`Description`, seam (`frank_dragoon_unit.bats`: pool law relaxed from
`pool == slots × ctx` to `ctx <= pool <= slots × ctx`, fetch-script name),
new runbook `frank-dragoon-flash-server-v2-native`, swift15 page re-banned
RETIRED, both indexes + AGENTS.md re-keyed, homepage card, `models.yml`
comment (numbers unchanged: window 217,088 / cap 45,056 — the per-REQUEST
ceiling did not move). OpenWebUI `frank-dragoon` row description re-keyed
(backup `webui.db.bak-20261017b`, no restart forced).
- Two stale claims fixed on the way: the retired-era rows said `200,192` ctx
(that ceiling was raised to 262,144 on 2026-10-07, inside the same era), and
the root index still called the halogen-27B page ACTIVE.
- Cost of the swap, stated plainly: two MAXIMUM-length conversations no longer
share the endpoint (one 262,144-position pool behind both slots), PPL +2.3 %,
thinking ×1.9. Bought: 7.4 GiB of RAM back as page cache for the table long
prompts read, +12.6 % cold prefill, retrieval 18/18, the base checkpoint
itself instead of the finetune.
## 2026-10-17 — frank-dragoon: ceiling and pool cut to 200,704; crawl reset fix
- Ask: lower frank-dragoon's context "to 196k". Settled on **200,704** for BOTH
`HALOGEN_CTX` and `HALOGEN_KV_POOL_POSITIONS` (49 × 4096 — the per-slot rung
frank-zealot runs; the card recommends pool == ctx, and that equality is where
the memory is). Nothing in the fleet needs the bigger window, and the v2
checkpoint serves anything up to its native 262,144.
- Boot ledger (`7c4dbeb`, `status=deployed`): `reserving the KV pool, 200704
positions (about 5.5 GiB)`, `62.9 GiB of weights … 5.5 of KV pool, 5.7 of
working memory, 74.1 GiB in all`, prompt cache entries 111 MiB at the new ctx,
no clamp line. My arithmetic predicted 5.4/74 — the ledger is quoted instead.
- Harness moved WITH the ceiling (400-zone law): `models.yml`
`contextWindow: 155648` = ceiling − 45,056; compaction now fires at 85 % =
132,301 est ≈ 136k measured, and the 180k compaction target of the 262,144 era
is unreachable because the window itself is smaller. `maxTokens` 45,056 and the
coupled cap/timeout 65536/7200 untouched. Worst-case harness request =
1.03 × est + min(45056, 151552 − est) ≤ 156.2k tokens vs the 200,704 ceiling.
- Probes on the live endpoint: over the ceiling → `400 … it is 297109 tokens and
the context is 200704, 96405 tokens over` in 0.86 s (preflight rejects before
any prefill, so probing costs the endpoint nothing); harness-shaped admission →
122,679-token prompt with `max_tokens: 45056` (reservation 167,735) accepted,
81.1 s wall (≈1,513 t/s cold prefill), `27*59` answered **1593** from inside a
122k-token haystack, `finish_reason: stop`.
- Seam re-keyed (`frank_dragoon_unit.bats`): the fit-law test now proves the
invariant at the new pair — `ctx ≤ pool ≤ slots × ctx` and
`harness window ≤ ctx − 45056` — so a future raise of one without the other
fails the gate. bats 371/371.
- Deploy verdict: the endpoint dispatched healthy (`status=deployed`, `/health`
200), but the `[3/3]` Playwright gate failed on `12-crawl` and the deployer
correctly said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was
Plex Web, signed out in the `qa-e2e` profile, running its own sign-in flow,
which broke the spec two different ways:
1. it redirects the top frame, so Chromium cancelled the crawl's in-loop
`page.goto("/")` with `net::ERR_ABORTED` — and that reset sat outside the
`try`, so one host aborted the whole crawl (once as a 420 s timeout, once at
5.4 s); fixed with `backHome()`, which re-issues on that one error class.
2. the same hand-off leaves the browser on `app.plex.tv` (Plex's hosted surface
for a session-less client), so the `hostname == target` law could never
settle — this is what the first full-suite re-run surfaced as `1 flaky`;
fixed with an explicit `HANDOFFS` landing list.
Plex itself was healthy throughout (container up 2 days; 32400 answers HTTPS
401 / plain-HTTP empty reply, which is what Plex Web authenticates against),
and the guard contract is still asserted for every vhost: challenge cleared, no
authentik `uidField`, status < 500. Both symptoms recorded as gotcha 6.
- Gate after both fixes: **33 passed (1.2 m)**, crawl green on the first attempt
(6.5 s) — the earlier `DEPLOY INCOMPLETE` is cleared, and the deploy marker
stays at `7c4dbeb` because every later commit is docs/tests only, which the
planner treats as a no-op for the service.
- Cost of the cut: two maximum-length conversations never coexist (one
200,704-position pool behind both slots; two of ≤100k still do). Bought: 1.7
GiB back as page cache for the 47.68 GiB n-gram table — 9.1 GiB against the
2026-10-06 shape's 524,288 pool — and a smaller prompt-cache entry.
+40 -5
View File
@@ -8,18 +8,53 @@ import { expect, test, type Page } from "@playwright/test";
const CHALLENGE = "Making sure you're not a bot"; const CHALLENGE = "Making sure you're not a bot";
async function settleThroughGuard(page: Page, targetHost: string): Promise<void> { // Vendor hand-offs. An app may legitimately carry a session-less browser to its
// OWN hosted surface: Plex Web (plex.franksdank.cloud, served by our stack)
// bounces a signed-out visitor to app.plex.tv after it has already answered.
// That is the app working, not a guard-chain failure — the fleet contract this
// spec owns is still judged: the Anubis challenge cleared, no authentik
// uidField prompt appeared, and the app answered below 500. Any host NOT listed
// here still has to land on its own vhost.
const HANDOFFS: Record<string, string[]> = {
"plex.franksdank.cloud": ["app.plex.tv", "accounts.plex.tv", "clients.plex.tv"],
};
async function settleThroughGuard(
page: Page,
targetHost: string,
foreignHosts: string[] = [],
): Promise<void> {
// Each guarded vhost runs its own Anubis proof-of-work on first visit; // Each guarded vhost runs its own Anubis proof-of-work on first visit;
// wait for the challenge to clear before judging the destination. // wait for the challenge to clear before judging the destination.
await expect await expect
.poll(async () => !(await page.locator("body").innerText()).includes(CHALLENGE), { timeout: 90_000 }) .poll(async () => !(await page.locator("body").innerText()).includes(CHALLENGE), { timeout: 90_000 })
.toBe(true); .toBe(true);
const landing = [targetHost, ...foreignHosts];
await expect await expect
.poll(() => new URL(page.url()).hostname, { timeout: 60_000 }) .poll(() => landing.includes(new URL(page.url()).hostname), { timeout: 60_000 })
.toBe(targetHost); .toBe(true);
expect(await page.locator('input[name="uidField"]').count()).toBe(0); expect(await page.locator('input[name="uidField"]').count()).toBe(0);
} }
async function backHome(page: Page): Promise<void> {
// The homepage is the loop's only anchor, and some apps keep issuing their
// OWN navigation after load — Plex Web's "would like to sign in to your
// Plex account" iframe redirects the top frame, which cancels our goto with
// net::ERR_ABORTED. That abort is the app's behaviour, not a guard-chain
// failure (the app itself was already judged above), so tolerate it and
// re-issue rather than letting one host abort the whole crawl.
for (let attempt = 0; attempt < 3; attempt++) {
try {
await page.goto("/", { waitUntil: "domcontentloaded" });
return;
} catch (err) {
if (!String(err).includes("ERR_ABORTED")) throw err;
await page.waitForTimeout(1_000);
}
}
await page.goto("/");
}
test("every user open link reaches its app", async ({ page }) => { test("every user open link reaches its app", async ({ page }) => {
test.setTimeout(420_000); test.setTimeout(420_000);
// Expected link count comes from the live registry, mirroring the // Expected link count comes from the live registry, mirroring the
@@ -39,7 +74,7 @@ test("every user open link reaches its app", async ({ page }) => {
const target = new URL(link.href); const target = new URL(link.href);
try { try {
await page.goto(link.href); await page.goto(link.href);
await settleThroughGuard(page, target.hostname); await settleThroughGuard(page, target.hostname, HANDOFFS[target.hostname] ?? []);
const resp = await page.request.get(page.url()); const resp = await page.request.get(page.url());
if (resp.status() >= 500) { if (resp.status() >= 500) {
failures.push(`${link.label}: ${page.url()} answered ${resp.status()}`); failures.push(`${link.label}: ${page.url()} answered ${resp.status()}`);
@@ -47,7 +82,7 @@ test("every user open link reaches its app", async ({ page }) => {
} catch (err) { } catch (err) {
failures.push(`${link.label}: ${err instanceof Error ? err.message : String(err)}`); failures.push(`${link.label}: ${err instanceof Error ? err.message : String(err)}`);
} }
await page.goto("/"); await backHome(page);
await expect(page.locator("#grid a.open")).toHaveCount(expectedUserLinks); await expect(page.locator("#grid a.open")).toHaveCount(expectedUserLinks);
} }
expect(failures, failures.join("\n")).toEqual([]); expect(failures, failures.join("\n")).toEqual([]);
+28 -27
View File
@@ -3,7 +3,7 @@
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the # of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2, # Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE # digest-pinned), driven by the root unit's serve script — 2 slots over ONE
# 262,144-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0) # 200,704-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune # and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06. # shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
# #
@@ -81,8 +81,8 @@ good_env() {
"FRANK_DRAGOON_VISION_BYTES=0" "FRANK_DRAGOON_VISION_BYTES=0"
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json" "FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
"HALOGEN_MODEL_ID=frank-dragoon" "HALOGEN_MODEL_ID=frank-dragoon"
"HALOGEN_CTX=262144" "HALOGEN_CTX=200704"
"HALOGEN_KV_POOL_POSITIONS=262144" "HALOGEN_KV_POOL_POSITIONS=200704"
"HALOGEN_KV_SLOTS=2" "HALOGEN_KV_SLOTS=2"
"HALOGEN_SPEC_ADAPT=0" "HALOGEN_SPEC_ADAPT=0"
"HALOGEN_MAX_TOK=8192" "HALOGEN_MAX_TOK=8192"
@@ -153,13 +153,13 @@ good_env() {
pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")" pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")"
mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")" mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")"
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")" reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
# The pool is ONE budget behind the slots (boot ledger: "2 slots over ONE # The pool is ONE budget behind the slots (boot ledger form: "2 slots over
# 262144-position KV pool, each request up to 262144"), NOT slots x ctx: # ONE <pool>-position KV pool, each request up to HALOGEN_CTX"), NOT
# the card ships it at one window and that IS the memory saving of this # slots x ctx — one window behind both slots IS the memory saving. What
# payload. What still binds is that the pool back ONE full-length # still binds: the pool must back ONE full-length request (or the
# request, or the advertised per-request ceiling would be unreachable — # advertised per-request ceiling would be unreachable, hence pool >= ctx),
# and that it stay inside unified memory, which the ledger sum below # it must not pretend to be slots x ctx (pool <= slots * ctx), and the
# checks. # ledger sum below must stay inside unified memory.
[ "$pool" -ge "$ctx" ] [ "$pool" -ge "$ctx" ]
[ "$pool" -le $(( slots * ctx )) ] [ "$pool" -le $(( slots * ctx )) ]
kv_gib=$(( pool * 28 * 1024 / 1073741824 )) kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
@@ -273,8 +273,8 @@ $(cat "$TEST_TMP/serve.log")"
[[ "$output" == *"--name frank-dragoon-halogen"* ]] [[ "$output" == *"--name frank-dragoon-halogen"* ]]
[[ "$output" == *"--publish 7770:8731"* ]] [[ "$output" == *"--publish 7770:8731"* ]]
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]] [[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]] [[ "$output" == *"-e HALOGEN_CTX=200704"* ]]
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=262144"* ]] [[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=200704"* ]]
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]] [[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]] [[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]] [[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
@@ -291,23 +291,24 @@ $(cat "$TEST_TMP/serve.log")"
} }
@test "the shipped env keeps the harness fit law inside the ceiling" { @test "the shipped env keeps the harness fit law inside the ceiling" {
# 262144 (the checkpoint's native window) is the per-slot hard ceiling # HALOGEN_CTX is the hard per-request ceiling (a longer prompt is a 400
# (a longer prompt is a 400 naming the limit, never a silent # naming the limit, never a silent truncation). The law is INVARIANT to
# truncation). The harness clamps max_tokens = min(45056, 217088 − est # the value: the harness window must sit ~45k UNDER it, because the
# − 4096), compacts at 85 % of its declared 217088 window, and its # harness clamps max_tokens = min(45056, window − est − 4096), compacts
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent # at 85 % of its declared window, and its chars/4 est UNDER-counts the
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with # rendered prompt by ~3 % on code/agent traffic (measured 4,450 tokens at
# window == ceiling, every request over ~146k est was a guaranteed # est 152,938 on 2026-10-07 — with window == ceiling, every request over
# 400 before compaction could fire at 85 %). The 45k gap between the # ~65 % of the window was a guaranteed 400 before compaction could fire).
# harness window and the engine ceiling must absorb that error at # The shipped pair is ctx 200704 / window 155648 (lowered from
# both monotone maxima of est + budget (est ≤ 217088 − 4096): # 262144/217088 on 2026-10-17); the gap must absorb the error at BOTH
# full budget at the compaction threshold: 1.03*184524 + 45056 # monotone maxima of est + budget (est ≤ window − 4096):
# clamped budget at the clamp's zero point: 1.03*212992 + 0 # full budget at the compaction threshold: 1.03*132300 + 45056
# clamped budget at the clamp's zero point: 1.03*151552 + 0
# and the compaction request that rides on frank-zealot (its # and the compaction request that rides on frank-zealot (its
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k). # 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
grep -qE '^HALOGEN_CTX=262144$' "$ENV_FILE" grep -qE '^HALOGEN_CTX=200704$' "$ENV_FILE"
ctx=262144 ctx=200704
hw=217088; mt=45056 hw=155648; mt=45056
thr=$(( hw * 85 / 100 )) thr=$(( hw * 85 / 100 ))
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ] [ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
zero=$(( hw - 4096 )) zero=$(( hw - 4096 ))