frank-dragoon: lower the context to 200704 (ctx and pool together)
Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung frank-zealot already runs). The checkpoint's native window is 262,144 and the engine serves anything below it, so this is a RAM decision, not a capability limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured 14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table loses -- the freed positions become page cache for the 47.68 GiB table that long prompts read. Engine-side total ~74 GiB instead of 75.8. The 400-zone law is invariant to the value, so the harness moves WITH the ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est (~136k measured); the 180k target of the 262,144 era is unreachable because the window itself is 155,648. maxTokens 45056 and the coupled cap/timeout 65536/7200 are untouched. Cost stated plainly: with pool == ctx, ONE maximum-length request fills the pool, so two long conversations never coexist (two of <=100k do). Acceptable because frank-dragoon carries only the slow role and the security-reviewer subagent, not the interactive workhorse. Seam re-keyed: the fit-law test now proves the invariant at the new pair, and the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
This commit is contained in:
@@ -66,9 +66,11 @@ ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b8680670
|
||||
|
||||
## Shape — the card's settings, and the three deliberate deviations
|
||||
|
||||
`HALOGEN_CTX=262144` (per-request ceiling), `HALOGEN_KV_SLOTS=2`,
|
||||
**`HALOGEN_KV_POOL_POSITIONS=262144`** (the card's recommendation, and the
|
||||
memory saving), `HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
||||
`HALOGEN_CTX=200704` (per-request ceiling — the card's 262,144 lowered the
|
||||
same evening, see below), `HALOGEN_KV_SLOTS=2`,
|
||||
**`HALOGEN_KV_POOL_POSITIONS=200704`** (one pool behind both slots; the card
|
||||
recommends it equal to ctx, and that equality is where the memory is),
|
||||
`HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
||||
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
|
||||
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
|
||||
`HALOGEN_HOST_RESERVE_GIB=20`.
|
||||
@@ -81,24 +83,28 @@ Deviations from the card, each on purpose:
|
||||
(the adaptive default disables the drafter per request and halves decode).
|
||||
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
|
||||
|
||||
**The pool is one budget, not a partition.** Boot ledger: `listening on
|
||||
127.0.0.1:8730, 2 slots over ONE 262144-position KV pool, each request up to
|
||||
262144`. The per-request ceiling is unchanged, so the harness contract
|
||||
(`contextWindow: 217088`) does not move — what changed is that **two
|
||||
maximum-length conversations can no longer share the endpoint**. That is
|
||||
acceptable here because frank-dragoon carries the `slow` (medium-thinking)
|
||||
role and the `security-reviewer` subagent, not the interactive workhorse
|
||||
(that is frank-archon).
|
||||
**The pool is one budget, not a partition.** Boot ledger form: `listening on
|
||||
127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to
|
||||
<HALOGEN_CTX>`. Both numbers were lowered to **200,704** the same evening
|
||||
(the card ships 262,144; Frank took the RAM — nothing else in the fleet asks
|
||||
for the bigger window), so `models.yml` `contextWindow` moved to **155,648**
|
||||
= ceiling − 45k with it. With pool == ctx, one maximum-length request fills
|
||||
the pool: **two long conversations can no longer share the endpoint** (two of
|
||||
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
|
||||
(medium-thinking) role and the `security-reviewer` subagent, not the
|
||||
interactive workhorse (that is frank-archon).
|
||||
|
||||
**Memory ledger** (121.2 GiB unified), from the boot ledger of the live
|
||||
candidate: `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
|
||||
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
|
||||
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
|
||||
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …`. The n-gram
|
||||
table rides as page cache — `mapped 51.2 GB, NOT pinned (Pin::None)` — which
|
||||
is exactly why cutting the pool is a *double* saving. FLAGS.md law: **RAM the
|
||||
pool takes is RAM that table loses**; the 7.4 GiB the smaller pool hands back
|
||||
is page cache for the 47.68 GiB table that long prompts read.
|
||||
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in
|
||||
all`. After the same-evening ctx/pool cut to 200,704 the reservation is ~5.4
|
||||
GiB and the total ~74. The n-gram table rides as page cache — `mapped 51.2 GB,
|
||||
NOT pinned (Pin::None)` — which is exactly why every pool cut is a *double*
|
||||
saving. FLAGS.md law: **RAM the pool takes is RAM that table loses**; the ~9
|
||||
GiB the two cuts together handed back is page cache for the 47.68 GiB table
|
||||
that long prompts read.
|
||||
|
||||
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||
|
||||
@@ -19,7 +19,7 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||
digest serving the NATIVE v2 conversion of the abliterated
|
||||
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||
262,144-position KV pool, vision ON)** and
|
||||
200,704-position KV pool at ctx 200704, vision ON)** and
|
||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||
@@ -54,7 +54,7 @@ gfx1151 iGPU 114 GiB GTT]
|
||||
E[flash-server container gfx1151, digest-pinned image
|
||||
frank-dragoon, port 7770]
|
||||
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||
ONE 262144-position KV pool, 2 slots, MTP always-on, image input]
|
||||
ONE 200704-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
|
||||
G[llama.cpp PrismML fork CUDA build
|
||||
frank-zealot, port 7771]
|
||||
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
||||
@@ -87,9 +87,10 @@ frank-zealot, port 7771]
|
||||
**hardlinked** beside it — the card publishes only the main file, and the
|
||||
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
|
||||
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
|
||||
Card-recommended `HALOGEN_KV_POOL_POSITIONS=262144`: **one pool behind two
|
||||
slots** — 7.2 GiB where the old shape spent 14.6, the per-request ceiling
|
||||
unchanged at 262,144, and the freed RAM becomes page cache for the n-gram
|
||||
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=200704`: **one pool
|
||||
behind two slots** (the card ships 262,144/262,144 = 7.2 GiB; Frank took the
|
||||
RAM the same evening — ~5.4 GiB, one maximum-length request filling the
|
||||
pool), and the freed RAM becomes page cache for the n-gram
|
||||
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
|
||||
|
||||
Reference in New Issue
Block a user