frank-dragoon: lower the context to 200704 (ctx and pool together)

Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and
HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung
frank-zealot already runs). The checkpoint's native window is 262,144 and the
engine serves anything below it, so this is a RAM decision, not a capability
limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured
14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table
loses -- the freed positions become page cache for the 47.68 GiB table that
long prompts read. Engine-side total ~74 GiB instead of 75.8.

The 400-zone law is invariant to the value, so the harness moves WITH the
ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping
the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs
the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est
(~136k measured); the 180k target of the 262,144 era is unreachable because
the window itself is 155,648. maxTokens 45056 and the coupled
cap/timeout 65536/7200 are untouched.

Cost stated plainly: with pool == ctx, ONE maximum-length request fills the
pool, so two long conversations never coexist (two of <=100k do). Acceptable
because frank-dragoon carries only the slow role and the security-reviewer
subagent, not the interactive workhorse.

Seam re-keyed: the fit-law test now proves the invariant at the new pair, and
the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
This commit is contained in:
Frank committed 2026-10-17 22:29:22 -07:00
1 parent 860386d283
commit 7c4dbebe24
8 files changed
+144 -123

No files matched your search

+6 -5
View File
@@ -19,7 +19,7 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
digest serving the NATIVE v2 conversion of the abliterated
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
262,144-position KV pool, vision ON)** and
200,704-position KV pool at ctx 200704, vision ON)** and
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
@@ -54,7 +54,7 @@ gfx1151 iGPU 114 GiB GTT]
E[flash-server container gfx1151, digest-pinned image
frank-dragoon, port 7770]
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
ONE 262144-position KV pool, 2 slots, MTP always-on, image input]
ONE 200704-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
G[llama.cpp PrismML fork CUDA build
frank-zealot, port 7771]
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
@@ -87,9 +87,10 @@ frank-zealot, port 7771]
**hardlinked** beside it — the card publishes only the main file, and the
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
Card-recommended `HALOGEN_KV_POOL_POSITIONS=262144`: **one pool behind two
slots** — 7.2 GiB where the old shape spent 14.6, the per-request ceiling
unchanged at 262,144, and the freed RAM becomes page cache for the n-gram
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=200704`: **one pool
behind two slots** (the card ships 262,144/262,144 = 7.2 GiB; Frank took the
RAM the same evening — ~5.4 GiB, one maximum-length request filling the
pool), and the freed RAM becomes page cache for the n-gram
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,