Frank
2b127163c8
frank-dragoon's total in-flight context was HALF frank-zealot's: zealot runs one 401,408-position pool split into two 200,704 conversations, while dragoon ran ONE 200,704-position pool behind its two slots, so a single maximum-length request filled it and two long conversations had to stay under ~100k each. HALOGEN_KV_POOL_POSITIONS 200704 -> 401408 = slots x ctx, the top of the seam's legal band (ctx <= pool <= slots * ctx). The PER-REQUEST ceiling does not move: HALOGEN_CTX stays 200704, because the checkpoint's native window is 262,144 and the engine's own 400 text refuses to exceed it without HALOGEN_ROPE_YARN. What this buys is concurrency, not a longer conversation, which is why the harness contextWindow (155648 = ceiling - 45056) is untouched. Cost: ~10.8 GiB of KV where 200,704 cost 5.5 (262,144 cost 7.2; the 2026-10-06 shape's 524,288 measured 14.6), taken from the page cache the 51.2 GB n-gram table reads through — the warm prompt-cache ratio is the number to re-measure after the boot. Engine-side total goes 74.1 -> ~79.4 GiB of 121.2. Also carries scripts/download-frank-zealot-bonsai2-ptq1.sh: the PTQ1_0 candidate payload fetcher (5,946,648,928 B, LFS sha256-verified, mmproj hardlinked), with the publisher's own throughput table in its header — the packing is smaller but the card measures it SLOWER on Ampere, so the swap is decided on our card, not on the card's A100 row.
32 lines
1.3 KiB
Desktop File
32 lines
1.3 KiB
Desktop File
[Unit]
|
|
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 401408-position pool at ctx 200704, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
# The engine pins staging buffers; keep the memlock budget unbounded so an
|
|
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
|
# other LLM units on this fleet.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
|
# systemd stop alone cannot stop the container child; stop it explicitly
|
|
# (frank-archon pattern). The engine drains on SIGTERM.
|
|
ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
|
|
TimeoutStartSec=1200
|
|
TimeoutStopSec=120
|
|
# always, not on-failure: an engine that exits 0 after an internal fault
|
|
# would otherwise leave the endpoint down until someone notices.
|
|
Restart=always
|
|
RestartSec=10
|
|
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
|
# on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
|
|
# stack; make it a late OOM candidate, not the first.
|
|
OOMScoreAdjust=-500
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|