docs(frank-dragoon): the 401,408-position pool is live; every figure re-keyed to the boot ledger
Deployed 2026-10-18 (e2e 33/33). The boot ledger prints `reserving the KV pool, 401408 positions (about 11.0 GiB)` and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`, so the ~10.8 GiB that every page quoted from the ~28 KiB/position arithmetic is replaced by the reservation the engine actually makes (11.0) and its preflight bill (11.3). Host MemAvailable fell 26 -> 22 GiB across the raise: the FLAGS.md collision (pool RAM is n-gram page cache) paid in full, so the warm prompt-cache ratio is flagged as the number to watch. All "committed but not yet deployed" clauses retired: AGENTS.md topology row, KV-pool law and memory ledger, frank-dragoon.env lore, both index pages, the iommu page, and the runbook chronology (third production boot recorded). The sweep also caught the LLM index still describing frank-zealot as Bonsai-2-27B at 2 x 200,704 in its intro and topology diagram - re-keyed to the live 9B: 8 full-attention + 24 Gated DeltaNet + blk.32 MTP, 3 x 262,144, tensor type 141.
This commit is contained in:
@@ -202,10 +202,9 @@ boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) +
|
||||
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
|
||||
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
|
||||
answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
|
||||
slot parity was COMMITTED — `2b12716` — but as of 2026-10-18 NOT yet
|
||||
redeployed: the live boot still reserves ONE 200,704-position pool (5.5 GiB),
|
||||
host `frank-dragoon.env` included. When it lands, the reservation is ~10.8
|
||||
GiB and the engine-side total ~79.4 GiB;
|
||||
slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
|
||||
boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
|
||||
is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
|
||||
see [[frank-dragoon-flash-server-v2-native]].)
|
||||
|
||||
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
|
||||
|
||||
@@ -94,9 +94,9 @@ the pool: **two long conversations can no longer share the endpoint** (two of
|
||||
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
|
||||
(medium-thinking) role and the `security-reviewer` subagent, not the
|
||||
interactive workhorse (that is frank-archon).
|
||||
Frank lifted exactly that concurrency limit on 2026-10-18 — see *Pool
|
||||
doubled for slot parity* below; that lift is COMMITTED but not yet deployed,
|
||||
so the sentences above remain the live truth until the next redeploy.
|
||||
Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is
|
||||
**deployed** (section below): the pool is again `slots × ctx`, so two
|
||||
maximum-length conversations fit the endpoint at once.
|
||||
|
||||
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
|
||||
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||
@@ -115,7 +115,7 @@ that long prompts read.
|
||||
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||
|
||||
## Pool doubled for slot parity — committed 2026-10-18, awaiting redeploy
|
||||
## Pool doubled for slot parity — committed AND deployed 2026-10-18
|
||||
|
||||
`HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's
|
||||
call: frank-dragoon's total in-flight context was **half** frank-zealot's —
|
||||
@@ -133,24 +133,28 @@ conversation had to stay under ~100k (commit message;
|
||||
(`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top
|
||||
of the legal band, and exactly frank-zealot's pool number — parity, not
|
||||
approximation.
|
||||
- Expected reservation **~10.8 GiB** at ~28 KiB/position (preflight formula,
|
||||
`scripts/frank-dragoon.env:70-72`); the measured points beside it are 5.5
|
||||
GiB at 200,704 (boot ledger 2026-10-18), 7.2 GiB reserved / 7.4 GiB billed
|
||||
at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
|
||||
- Measured reservation **11.0 GiB** — the boot ledger prints `reserving the KV
|
||||
pool, 401408 positions (about 11.0 GiB)`. The preflight KV-budget line bills
|
||||
**11.3 GiB** (`~28 KiB/position incl. block scratch + ~115 MiB/slot`) plus
|
||||
1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704,
|
||||
7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
|
||||
(`scripts/frank-dragoon.env:49-52`, `:74-76`).
|
||||
- That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the
|
||||
pool takes is RAM that table loses.
|
||||
|
||||
**Deploy state as of 2026-10-18: NOT yet deployed.** The live container
|
||||
(booted 2026-10-18 01:25) still runs `HALOGEN_KV_POOL_POSITIONS=200704` —
|
||||
verified in its `flash_serve` `/proc/<pid>/environ` and in the host-deployed
|
||||
`frank-dragoon.env` — and the 2026-10-18 boot ledger reserves `the KV pool,
|
||||
200704 positions (about 5.5 GiB)` with `74.1 GiB in all`
|
||||
(`/home/dank/logs/frank_dragoon.log`). Until the next
|
||||
`./scripts/deploy.sh dankclaw --service frank-dragoon`, the boot ledger keeps
|
||||
printing `2 slots over ONE 200704-position KV pool`; the first boot at the
|
||||
new pool must show `ONE 401408-position KV pool` and the ~10.8 GiB
|
||||
reservation — read that line before quoting 401,408 as live anywhere.
|
||||
**Deployed 2026-10-18** (`./scripts/deploy.sh dankclaw --service
|
||||
frank-dragoon`, Playwright gate 33/33). The boot ledger confirms the shape:
|
||||
`reserving the KV pool, 401408 positions (about 11.0 GiB)`,
|
||||
`2 slots over ONE 401408-position KV pool, each request up to 200704 (it
|
||||
reserves prompt + max_tokens)`, and the sum
|
||||
`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
|
||||
of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12
|
||||
entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram
|
||||
47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable`
|
||||
fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid
|
||||
in page cache. **Not re-measured since the raise:** the warm prompt-cache ratio
|
||||
(136.7× on the pre-raise boot) — that is the number to watch if a long warm
|
||||
turn starts costing full prefill again.
|
||||
|
||||
## Measured A/B — same battery, both boots (2026-10-17)
|
||||
|
||||
@@ -307,9 +311,17 @@ restart; `webui.db.bak-20261017b` holds the pre-cutover row).
|
||||
**What the cut cost:** with pool == ctx one maximum-length request fills the
|
||||
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
|
||||
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
|
||||
is unreachable because the window itself is smaller. (The pool == ctx
|
||||
concurrency limit is superseded — on redeploy only — by *Pool doubled for slot
|
||||
parity* above; the window math stands, the pool never moves the ceiling.)
|
||||
is unreachable because the window itself is smaller. (That pool == ctx
|
||||
concurrency limit IS superseded: the doubling deployed the next day — see
|
||||
*Pool doubled for slot parity* above. The window math stands, though: the pool
|
||||
never moves the per-request ceiling.)
|
||||
**What it bought:** 1.7
|
||||
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
|
||||
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
|
||||
|
||||
**Third production boot — the `401408` pool, deployed 2026-10-18.** Same ctx,
|
||||
same working set, same 111 MiB prompt-cache entry; the pool line reads
|
||||
`reserving the KV pool, 401408 positions (about 11.0 GiB)` and the total
|
||||
`79.6 GiB in all`. Ledger and the measured cost in
|
||||
*Pool doubled for slot parity* above; host `MemAvailable` after that boot:
|
||||
22 GiB.
|
||||
|
||||
@@ -22,14 +22,15 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||
digest serving the NATIVE v2 conversion of the abliterated
|
||||
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||
200,704-position KV pool at ctx 200704 — the 2 × ctx = 401,408 raise is
|
||||
committed 2026-10-18 but NOT yet deployed, vision ON)** and
|
||||
401,408-position KV pool — 2 × ctx, raised and DEPLOYED 2026-10-18 — at ctx
|
||||
200704, vision ON)** and
|
||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||
Ternary-Bonsai-2-9B at **PQ2_0** — 9B qwen35 hybrid (8 full-attention + 24
|
||||
Gated DeltaNet layers + the blk.32 MTP block), THREE 262,144-token
|
||||
conversations, MTP present in the file but OFF by measurement,
|
||||
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
|
||||
the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom
|
||||
tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the
|
||||
the publisher's own Q8_0 mmproj, on the RTX 3090 — and the custom
|
||||
tensor type 141 + `prism.hadamard.*` metadata mean no other loader can open the
|
||||
file)**.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
@@ -58,11 +59,11 @@ gfx1151 iGPU 114 GiB GTT]
|
||||
E[flash-server container gfx1151, digest-pinned image
|
||||
frank-dragoon, port 7770]
|
||||
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||
ONE 200704-position KV pool at ctx 200704 (401408 committed, awaiting deploy), 2 slots, MTP always-on, image input]
|
||||
ONE 401408-position KV pool at ctx 200704 = 2 slots x ctx, MTP always-on, image input]
|
||||
G[llama.cpp PrismML fork CUDA build
|
||||
frank-zealot, port 7771]
|
||||
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
||||
2 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
||||
H[Ternary-Bonsai-2-9B PQ2_0 GGUF + Q8_0 mmproj
|
||||
3 x 262144 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
||||
E2 --> E --> F
|
||||
E2 --> G --> H
|
||||
end
|
||||
@@ -94,13 +95,10 @@ frank-zealot, port 7771]
|
||||
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**):
|
||||
**one pool behind two slots**, sized so TWO maximum-length requests coexist —
|
||||
raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total
|
||||
in-flight context, ~10.8 GiB reserved against the card's 262,144/262,144 =
|
||||
7.2 GiB. **Deploy state 2026-10-18: the running container and the
|
||||
host-deployed env still carry the 200,704-position pool** (boot ledger
|
||||
`reserving the KV pool, 200704 positions (about 5.5 GiB)`; host
|
||||
`frank-dragoon.env` `HALOGEN_KV_POOL_POSITIONS=200704`) — the doubling lands
|
||||
on the next redeploy (commit `2b12716`).
|
||||
longer window: the per-request ceiling stays 200,704, because 262,144 is the
|
||||
in-flight context and **deployed the same evening**: the boot ledger reserves
|
||||
`401408 positions (about 11.0 GiB)` with `79.6 GiB in all`
|
||||
(`scripts/frank-dragoon.env:70-75`). It bought CONCURRENCY, not a longer
|
||||
window: the per-request ceiling stays 200,704, because 262,144 is the
|
||||
native wall and the engine refuses more without `HALOGEN_ROPE_YARN`.
|
||||
MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||
|
||||
Reference in New Issue
Block a user