frank-dragoon: lower the context to 200704 (ctx and pool together)
Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung frank-zealot already runs). The checkpoint's native window is 262,144 and the engine serves anything below it, so this is a RAM decision, not a capability limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured 14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table loses -- the freed positions become page cache for the 47.68 GiB table that long prompts read. Engine-side total ~74 GiB instead of 75.8. The 400-zone law is invariant to the value, so the harness moves WITH the ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est (~136k measured); the 180k target of the 262,144 era is unreachable because the window itself is 155,648. maxTokens 45056 and the coupled cap/timeout 65536/7200 are untouched. Cost stated plainly: with pool == ctx, ONE maximum-length request fills the pool, so two long conversations never coexist (two of <=100k do). Acceptable because frank-dragoon carries only the slow role and the security-reviewer subagent, not the interactive workhorse. Seam re-keyed: the fit-law test now proves the invariant at the new pair, and the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
This commit is contained in:
Frank committed
1 parent
860386d283
commit
7c4dbebe24
8 files changed
+144
-123
No files matched your search
@@ -3,7 +3,7 @@
|
||||
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
|
||||
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
|
||||
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE
|
||||
# 262,144-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
|
||||
# 200,704-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
|
||||
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
|
||||
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
|
||||
#
|
||||
@@ -81,8 +81,8 @@ good_env() {
|
||||
"FRANK_DRAGOON_VISION_BYTES=0"
|
||||
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
|
||||
"HALOGEN_MODEL_ID=frank-dragoon"
|
||||
"HALOGEN_CTX=262144"
|
||||
"HALOGEN_KV_POOL_POSITIONS=262144"
|
||||
"HALOGEN_CTX=200704"
|
||||
"HALOGEN_KV_POOL_POSITIONS=200704"
|
||||
"HALOGEN_KV_SLOTS=2"
|
||||
"HALOGEN_SPEC_ADAPT=0"
|
||||
"HALOGEN_MAX_TOK=8192"
|
||||
@@ -153,13 +153,13 @@ good_env() {
|
||||
pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||
mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||
# The pool is ONE budget behind the slots (boot ledger: "2 slots over ONE
|
||||
# 262144-position KV pool, each request up to 262144"), NOT slots x ctx:
|
||||
# the card ships it at one window and that IS the memory saving of this
|
||||
# payload. What still binds is that the pool back ONE full-length
|
||||
# request, or the advertised per-request ceiling would be unreachable —
|
||||
# and that it stay inside unified memory, which the ledger sum below
|
||||
# checks.
|
||||
# The pool is ONE budget behind the slots (boot ledger form: "2 slots over
|
||||
# ONE <pool>-position KV pool, each request up to HALOGEN_CTX"), NOT
|
||||
# slots x ctx — one window behind both slots IS the memory saving. What
|
||||
# still binds: the pool must back ONE full-length request (or the
|
||||
# advertised per-request ceiling would be unreachable, hence pool >= ctx),
|
||||
# it must not pretend to be slots x ctx (pool <= slots * ctx), and the
|
||||
# ledger sum below must stay inside unified memory.
|
||||
[ "$pool" -ge "$ctx" ]
|
||||
[ "$pool" -le $(( slots * ctx )) ]
|
||||
kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
|
||||
@@ -273,8 +273,8 @@ $(cat "$TEST_TMP/serve.log")"
|
||||
[[ "$output" == *"--name frank-dragoon-halogen"* ]]
|
||||
[[ "$output" == *"--publish 7770:8731"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=262144"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CTX=200704"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=200704"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
|
||||
@@ -291,23 +291,24 @@ $(cat "$TEST_TMP/serve.log")"
|
||||
}
|
||||
|
||||
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
||||
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
||||
# (a longer prompt is a 400 naming the limit, never a silent
|
||||
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
||||
# − 4096), compacts at 85 % of its declared 217088 window, and its
|
||||
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
|
||||
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with
|
||||
# window == ceiling, every request over ~146k est was a guaranteed
|
||||
# 400 before compaction could fire at 85 %). The 45k gap between the
|
||||
# harness window and the engine ceiling must absorb that error at
|
||||
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
|
||||
# full budget at the compaction threshold: 1.03*184524 + 45056
|
||||
# clamped budget at the clamp's zero point: 1.03*212992 + 0
|
||||
# HALOGEN_CTX is the hard per-request ceiling (a longer prompt is a 400
|
||||
# naming the limit, never a silent truncation). The law is INVARIANT to
|
||||
# the value: the harness window must sit ~45k UNDER it, because the
|
||||
# harness clamps max_tokens = min(45056, window − est − 4096), compacts
|
||||
# at 85 % of its declared window, and its chars/4 est UNDER-counts the
|
||||
# rendered prompt by ~3 % on code/agent traffic (measured 4,450 tokens at
|
||||
# est 152,938 on 2026-10-07 — with window == ceiling, every request over
|
||||
# ~65 % of the window was a guaranteed 400 before compaction could fire).
|
||||
# The shipped pair is ctx 200704 / window 155648 (lowered from
|
||||
# 262144/217088 on 2026-10-17); the gap must absorb the error at BOTH
|
||||
# monotone maxima of est + budget (est ≤ window − 4096):
|
||||
# full budget at the compaction threshold: 1.03*132300 + 45056
|
||||
# clamped budget at the clamp's zero point: 1.03*151552 + 0
|
||||
# and the compaction request that rides on frank-zealot (its
|
||||
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
|
||||
grep -qE '^HALOGEN_CTX=262144$' "$ENV_FILE"
|
||||
ctx=262144
|
||||
hw=217088; mt=45056
|
||||
grep -qE '^HALOGEN_CTX=200704$' "$ENV_FILE"
|
||||
ctx=200704
|
||||
hw=155648; mt=45056
|
||||
thr=$(( hw * 85 / 100 ))
|
||||
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
|
||||
zero=$(( hw - 4096 ))
|
||||
|
||||
Reference in new issue
Block a user