docs(frank-dragoon): the 401,408-position pool is live; every figure re-keyed to the boot ledger

Deployed 2026-10-18 (e2e 33/33). The boot ledger prints
`reserving the KV pool, 401408 positions (about 11.0 GiB)` and
`62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of working memory,
79.6 GiB in all`, so the ~10.8 GiB that every page quoted from the
~28 KiB/position arithmetic is replaced by the reservation the engine actually
makes (11.0) and its preflight bill (11.3). Host MemAvailable fell 26 -> 22 GiB
across the raise: the FLAGS.md collision (pool RAM is n-gram page cache) paid in
full, so the warm prompt-cache ratio is flagged as the number to watch.

All "committed but not yet deployed" clauses retired: AGENTS.md topology row,
KV-pool law and memory ledger, frank-dragoon.env lore, both index pages, the
iommu page, and the runbook chronology (third production boot recorded).

The sweep also caught the LLM index still describing frank-zealot as
Bonsai-2-27B at 2 x 200,704 in its intro and topology diagram - re-keyed to the
live 9B: 8 full-attention + 24 Gated DeltaNet + blk.32 MTP, 3 x 262,144,
tensor type 141.
This commit is contained in:
Frank
2026-10-18 13:33:04 -07:00
parent 695a5f3281
commit c0759492e6
7 changed files with 115 additions and 64 deletions
+11 -9
View File
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 on 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; ~10.8 GiB reserved where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 and DEPLOYED 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; **11.0 GiB reserved, measured on that deploy** (the preflight formula bills 11.3) where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 | | Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
Retired endpoints (`frank-npu`, and every prior runtime) are in Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -407,10 +407,10 @@ MAXIMUM-length conversations DO coexist. The PER-REQUEST ceiling stays
`HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's `HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's
own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context" own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context"
bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒ bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒
the ledger reserves "about 10.8 GiB", where 200,704 cost 5.5, 262,144 cost 7.2 the 2026-10-18 deploy RESERVES **11.0 GiB** (`reserving the KV pool, 401408
and the 2026-10-06 shape's 524,288 measured 14.6; the engine's preflight FORMULA positions (about 11.0 GiB)`; the preflight formula bills 11.3 with block scratch
bills slightly more than the reservation (block scratch + ~115 MiB/slot of O(1) + ~115 MiB/slot of O(1) state), where 200,704 cost 5.5, 262,144 cost 7.2 and
state) — quote the reservation. the 2026-10-06 shape's 524,288 measured 14.6 — quote the reservation.
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` + harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table). `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
@@ -428,10 +428,12 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
"video". "video".
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights **Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights
62.9 (trunk 62.11 + vision 0.84) + KV ~10.8 + working 5.7 62.9 (trunk 62.11 + vision 0.84) + KV **11.0** + working 5.7
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ **79.4 GiB in all**, the ~1.0 GiB prompt cache (`3.8 + 8.4 × MAX_TOK/16384`) = **79.6 GiB in all** (boot ledger, the
booked separately by the ledger, + `HALOGEN_HOST_RESERVE_GIB=20` → ~22 GiB 401,408-position deploy of 2026-10-18), the **1.3 GiB** prompt cache (12 entries
margin over baseline co-tenancy (the pool-equals-ctx boot printed 74.1 GiB with × 111 MiB at ctx 200704) booked separately by the ledger, + `HALOGEN_HOST_
RESERVE_GIB=20` → ~20 GiB margin over baseline co-tenancy (`MemAvailable` read
22 GiB right after the raise; the pool-equals-ctx boot printed 74.1 GiB with
~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside ~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside
the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why
every pool position is a double cost: **RAM the pool takes is RAM that table every pool position is a double cost: **RAM the pool takes is RAM that table
+15 -12
View File
@@ -68,10 +68,11 @@
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the # ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
# checkpoint's native window is 262,144 and the engine refuses to go past it # checkpoint's native window is 262,144 and the engine refuses to go past it
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position # without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
# (FLAGS.md) ⇒ ~10.8 GiB, where pool == ctx (200,704) cost 5.5, 262,144 cost # (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
# 7.2 and the 2026-10-06 shape's 524,288 measured 14.6. The engine's preflight # KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
# formula bills a little more than the reservation (block scratch + ~115 # cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
# MiB/slot of O(1) state) — quote the RESERVATION. FLAGS.md law: "RAM the pool # The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
# state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this # takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
# much page cache, so the warm prompt-cache ratio is what to re-measure. # much page cache, so the warm prompt-cache ratio is what to re-measure.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP # HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
@@ -82,14 +83,16 @@
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is # Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot # mmap'd page cache beside the checkpoint, not a pinned charge — the boot
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"): # ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~10.8 + # weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 79.4 GiB engine-side, with the # working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
# prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked # ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
# separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host # prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
# honest. Against the 121.2 GiB box that leaves ~22 GiB over the reserve for # the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
# baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot # the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
# ledger is the referee — read it after deploy and re-size if the engine # read 22 GiB right after the raise) for baseline co-tenancy
# prints a clamp. # (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
# referee — read it after deploy and re-size if the engine prints a
# clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy # HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
# turns the MTP head off for whole requests after low acceptance and # turns the MTP head off for whole requests after low acceptance and
# HALVES decode as a result. Frank accepted the head always-on trade # HALVES decode as a result. Frank accepted the head always-on trade
@@ -202,10 +202,9 @@ boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) +
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
answered 200. (Later the same day the pool DOUBLING to 401,408 positions for answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
slot parity was COMMITTED — `2b12716` — but as of 2026-10-18 NOT yet slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
redeployed: the live boot still reserves ONE 200,704-position pool (5.5 GiB), boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
host `frank-dragoon.env` included. When it lands, the reservation is ~10.8 is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
GiB and the engine-side total ~79.4 GiB;
see [[frank-dragoon-flash-server-v2-native]].) see [[frank-dragoon-flash-server-v2-native]].)
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/ **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
@@ -94,9 +94,9 @@ the pool: **two long conversations can no longer share the endpoint** (two of
≤100k can). That is acceptable here because frank-dragoon carries the `slow` ≤100k can). That is acceptable here because frank-dragoon carries the `slow`
(medium-thinking) role and the `security-reviewer` subagent, not the (medium-thinking) role and the `security-reviewer` subagent, not the
interactive workhorse (that is frank-archon). interactive workhorse (that is frank-archon).
Frank lifted exactly that concurrency limit on 2026-10-18 — see *Pool Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is
doubled for slot parity* below; that lift is COMMITTED but not yet deployed, **deployed** (section below): the pool is again `slots × ctx`, so two
so the sentences above remain the live truth until the next redeploy. maximum-length conversations fit the endpoint at once.
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot **Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk, printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
@@ -115,7 +115,7 @@ that long prompts read.
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk `HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
stays pinned. Disk-backing the weights is not a lever on this format. stays pinned. Disk-backing the weights is not a lever on this format.
## Pool doubled for slot parity — committed 2026-10-18, awaiting redeploy ## Pool doubled for slot parity — committed AND deployed 2026-10-18
`HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's `HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's
call: frank-dragoon's total in-flight context was **half** frank-zealot's — call: frank-dragoon's total in-flight context was **half** frank-zealot's —
@@ -133,24 +133,28 @@ conversation had to stay under ~100k (commit message;
(`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top (`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top
of the legal band, and exactly frank-zealot's pool number — parity, not of the legal band, and exactly frank-zealot's pool number — parity, not
approximation. approximation.
- Expected reservation **~10.8 GiB** at ~28 KiB/position (preflight formula, - Measured reservation **11.0 GiB** — the boot ledger prints `reserving the KV
`scripts/frank-dragoon.env:70-72`); the measured points beside it are 5.5 pool, 401408 positions (about 11.0 GiB)`. The preflight KV-budget line bills
GiB at 200,704 (boot ledger 2026-10-18), 7.2 GiB reserved / 7.4 GiB billed **11.3 GiB** (`~28 KiB/position incl. block scratch + ~115 MiB/slot`) plus
at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288 1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704,
7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
(`scripts/frank-dragoon.env:49-52`, `:74-76`). (`scripts/frank-dragoon.env:49-52`, `:74-76`).
- That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the - That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the
pool takes is RAM that table loses. pool takes is RAM that table loses.
**Deploy state as of 2026-10-18: NOT yet deployed.** The live container **Deployed 2026-10-18** (`./scripts/deploy.sh dankclaw --service
(booted 2026-10-18 01:25) still runs `HALOGEN_KV_POOL_POSITIONS=200704` — frank-dragoon`, Playwright gate 33/33). The boot ledger confirms the shape:
verified in its `flash_serve` `/proc/<pid>/environ` and in the host-deployed `reserving the KV pool, 401408 positions (about 11.0 GiB)`,
`frank-dragoon.env` — and the 2026-10-18 boot ledger reserves `the KV pool, `2 slots over ONE 401408-position KV pool, each request up to 200704 (it
200704 positions (about 5.5 GiB)` with `74.1 GiB in all` reserves prompt + max_tokens)`, and the sum
(`/home/dank/logs/frank_dragoon.log`). Until the next `62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
`./scripts/deploy.sh dankclaw --service frank-dragoon`, the boot ledger keeps of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12
printing `2 slots over ONE 200704-position KV pool`; the first boot at the entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram
new pool must show `ONE 401408-position KV pool` and the ~10.8 GiB 47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable`
reservation — read that line before quoting 401,408 as live anywhere. fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid
in page cache. **Not re-measured since the raise:** the warm prompt-cache ratio
(136.7× on the pre-raise boot) — that is the number to watch if a long warm
turn starts costing full prefill again.
## Measured A/B — same battery, both boots (2026-10-17) ## Measured A/B — same battery, both boots (2026-10-17)
@@ -307,9 +311,17 @@ restart; `webui.db.bak-20261017b` holds the pre-cutover row).
**What the cut cost:** with pool == ctx one maximum-length request fills the **What the cut cost:** with pool == ctx one maximum-length request fills the
pool, so two 200k conversations never coexist (two of ≤100 k do), and the pool, so two 200k conversations never coexist (two of ≤100 k do), and the
harness window is now 155,648 — the 180 k compaction target of the 262,144 era harness window is now 155,648 — the 180 k compaction target of the 262,144 era
is unreachable because the window itself is smaller. (The pool == ctx is unreachable because the window itself is smaller. (That pool == ctx
concurrency limit is superseded — on redeploy only — by *Pool doubled for slot concurrency limit IS superseded: the doubling deployed the next day — see
parity* above; the window math stands, the pool never moves the ceiling.) *Pool doubled for slot parity* above. The window math stands, though: the pool
never moves the per-request ceiling.)
**What it bought:** 1.7 **What it bought:** 1.7
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry. 2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
**Third production boot — the `401408` pool, deployed 2026-10-18.** Same ctx,
same working set, same 111 MiB prompt-cache entry; the pool line reads
`reserving the KV pool, 401408 positions (about 11.0 GiB)` and the total
`79.6 GiB in all`. Ledger and the measured cost in
*Pool doubled for slot parity* above; host `MemAvailable` after that boot:
22 GiB.
+14 -16
View File
@@ -22,14 +22,15 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
digest serving the NATIVE v2 conversion of the abliterated digest serving the NATIVE v2 conversion of the abliterated
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine, Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
200,704-position KV pool at ctx 200704 — the 2 × ctx = 401,408 raise is 401,408-position KV pool — 2 × ctx, raised and DEPLOYED 2026-10-18 — at ctx
committed 2026-10-18 but NOT yet deployed, vision ON)** and 200704, vision ON)** and
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml **llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48 Ternary-Bonsai-2-9B at **PQ2_0** — 9B qwen35 hybrid (8 full-attention + 24
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled, Gated DeltaNet layers + the blk.32 MTP block), THREE 262,144-token
conversations, MTP present in the file but OFF by measurement,
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom the publisher's own Q8_0 mmproj, on the RTX 3090 — and the custom
tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the tensor type 141 + `prism.hadamard.*` metadata mean no other loader can open the
file)**. file)**.
> 2026-08-22: the former `vllmScripts` repository is merged into the > 2026-08-22: the former `vllmScripts` repository is merged into the
@@ -58,11 +59,11 @@ gfx1151 iGPU 114 GiB GTT]
E[flash-server container gfx1151, digest-pinned image E[flash-server container gfx1151, digest-pinned image
frank-dragoon, port 7770] frank-dragoon, port 7770]
F[flash-server v2 native abliterated .hgn + ngram table + vision tower F[flash-server v2 native abliterated .hgn + ngram table + vision tower
ONE 200704-position KV pool at ctx 200704 (401408 committed, awaiting deploy), 2 slots, MTP always-on, image input] ONE 401408-position KV pool at ctx 200704 = 2 slots x ctx, MTP always-on, image input]
G[llama.cpp PrismML fork CUDA build G[llama.cpp PrismML fork CUDA build
frank-zealot, port 7771] frank-zealot, port 7771]
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj H[Ternary-Bonsai-2-9B PQ2_0 GGUF + Q8_0 mmproj
2 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision] 3 x 262144 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
E2 --> E --> F E2 --> E --> F
E2 --> G --> H E2 --> G --> H
end end
@@ -94,13 +95,10 @@ frank-zealot, port 7771]
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**): `HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**):
**one pool behind two slots**, sized so TWO maximum-length requests coexist — **one pool behind two slots**, sized so TWO maximum-length requests coexist —
raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total
in-flight context, ~10.8 GiB reserved against the card's 262,144/262,144 = in-flight context and **deployed the same evening**: the boot ledger reserves
7.2 GiB. **Deploy state 2026-10-18: the running container and the `401408 positions (about 11.0 GiB)` with `79.6 GiB in all`
host-deployed env still carry the 200,704-position pool** (boot ledger (`scripts/frank-dragoon.env:70-75`). It bought CONCURRENCY, not a longer
`reserving the KV pool, 200704 positions (about 5.5 GiB)`; host window: the per-request ceiling stays 200,704, because 262,144 is the
`frank-dragoon.env` `HALOGEN_KV_POOL_POSITIONS=200704`) — the doubling lands
on the next redeploy (commit `2b12716`).
longer window: the per-request ceiling stays 200,704, because 262,144 is the
native wall and the engine refuses more without `HALOGEN_ROPE_YARN`. native wall and the engine refuses more without `HALOGEN_ROPE_YARN`.
MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5 65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
+3 -2
View File
@@ -314,8 +314,9 @@ The LLM inference fleet on the two hosts. Full index:
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2 since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
abliterated** conversion of orcarouter's pinned BF16 abliterated** conversion of orcarouter's pinned BF16
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a (`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a
**401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, ~10.8 GiB — one **401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, 11.0 GiB
pool behind both slots, raised to 2 × ctx on 2026-10-18 so TWO maximum-length reserved — one pool behind both slots, raised to 2 × ctx and deployed
2026-10-18 so TWO maximum-length
requests coexist) and the stock peonist n-gram/vision companions **hardlinked** requests coexist) and the stock peonist n-gram/vision companions **hardlinked**
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
always-on, vision ON, no server-side sampler change always-on, vision ON, no server-side sampler change
+36
View File
@@ -2190,3 +2190,39 @@ every restart since the cutover.
200,704). Note the columns are `params`/`meta` JSON — the table has no 200,704). Note the columns are `params`/`meta` JSON — the table has no
`context_length`/`description` columns. `context_length`/`description` columns.
## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot
- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33. The
drop-in check the zealot incident bought ran FIRST: `DropInPaths` empty, so
the repo env governed. Boot ledger: `reserving the KV pool, 401408 positions
(about 11.0 GiB)`, `2 slots over ONE 401408-position KV pool, each request up
to 200704`, and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of
working memory, 79.6 GiB in all`; prompt cache ON (12 entries × 111 MiB at
ctx 200704 = 1.3 GiB), n-gram 47.68 GiB still `Pin::None`, no clamp line.
- The price is exactly the FLAGS.md collision: host `MemAvailable` fell
**26 → 22 GiB**, because every pool position is RAM the n-gram table's page
cache loses. The warm prompt-cache ratio (136.7× measured pre-raise) was NOT
re-measured — recorded in the runbook as the number to watch.
- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the
engine reserves **11.0 GiB** and its preflight bills **11.3**. Re-keyed in
AGENTS.md (topology row, KV-pool law, memory ledger), the `frank-dragoon.env`
lore block, both index pages, the iommu page and the runbook, and every
"committed but not yet deployed" clause was retired with the measurement.
- The same sweep caught the LLM index still describing frank-zealot as
**Bonsai-2-27B at 2 × 200,704** in its intro paragraph and topology diagram —
the 9B swap of the same day had not reached those two spots. Re-keyed: 8
full-attention + 24 Gated DeltaNet + the `blk.32` MTP block, 3 × 262,144,
tensor type **141**.
- PTQ1_0 provenance corrected: `scripts/download-frank-zealot-bonsai2-ptq1.sh`
IS committed, so the armed-but-never-measured arm that hijacked the zealot
deploy was a **repo-tracked candidate**, not an orphan. The runbook now states
the decision it still owes: measure it in a real window, or record it
declined.
- OpenWebUI: both decoration rows re-keyed in the live DB. The values live
inside JSON columns — `params.context.length` and `meta.description` — not in
fields named for them, and there is no `sqlite3` binary on the host, so
`python3` stdlib + `Connection.backup()` (`webui.db.bak-20261018`). Zealot
200,704 → **262,144** with the 9B description; dragoon 262,144 → **200,704**,
which had kept the pre-lowering window while its own description said
200,704.