Frank
7c4dbebe24
Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung frank-zealot already runs). The checkpoint's native window is 262,144 and the engine serves anything below it, so this is a RAM decision, not a capability limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured 14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table loses -- the freed positions become page cache for the 47.68 GiB table that long prompts read. Engine-side total ~74 GiB instead of 75.8. The 400-zone law is invariant to the value, so the harness moves WITH the ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est (~136k measured); the 180k target of the 262,144 era is unreachable because the window itself is 155,648. maxTokens 45056 and the coupled cap/timeout 65536/7200 are untouched. Cost stated plainly: with pool == ctx, ONE maximum-length request fills the pool, so two long conversations never coexist (two of <=100k do). Acceptable because frank-dragoon carries only the slow role and the security-reviewer subagent, not the interactive workhorse. Seam re-keyed: the fit-law test now proves the invariant at the new pair, and the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
32 lines
1.3 KiB
Desktop File
32 lines
1.3 KiB
Desktop File
[Unit]
|
|
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 200704-position pool at ctx 200704, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
# The engine pins staging buffers; keep the memlock budget unbounded so an
|
|
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
|
# other LLM units on this fleet.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
|
# systemd stop alone cannot stop the container child; stop it explicitly
|
|
# (frank-archon pattern). The engine drains on SIGTERM.
|
|
ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
|
|
TimeoutStartSec=1200
|
|
TimeoutStopSec=120
|
|
# always, not on-failure: an engine that exits 0 after an internal fault
|
|
# would otherwise leave the endpoint down until someone notices.
|
|
Restart=always
|
|
RestartSec=10
|
|
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
|
# on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
|
|
# stack; make it a late OOM candidate, not the first.
|
|
OOMScoreAdjust=-500
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|