frank-large: reserve 15 -> 16 for co-tenant headroom

Operator asked for 18. 18 is not available: the engine's own fit-check line,
read from a clamped boot rather than modelled, requires budget >= 36.5 GiB to
keep 524288 at max_tok 32768 ("524288 positions need ~35.0 GiB (plus 1.5 of
margin)"). budget(18) = 35.5, so 18 silently un-doubles the pool we just paid
the reserve for. 17 fits by 0.02, inside the noise of the 67.7 resident-weights
term. 16 is the largest reserve that survives the check with real margin:
37.5 budget vs 36.5 required.

Recorded the ceiling and both remedies in the unit comment. The reserve is now
spent, so a future stall is answered by the pool or the arena, not here.
This commit is contained in:
Frank
2026-09-12 21:27:58 -07:00
parent 2969e08616
commit b87f18569a
@@ -125,12 +125,16 @@ Environment=HALOGEN_KV_SLOTS=4
# — raise the reserve back to 24 and the pool silently clamps to 262144 again.
Environment=HALOGEN_MAX_TOK=32768
# RAM left for the n-gram table the engine reads through the page cache.
# LOWERED 2026-09-12 from 24 (vendor default is 20) to 15 to buy the doubled
# pool at full prefill speed:
# 121.2 host - 67.7 weights - 15 reserve = 38.5 budget
# need = 14.4 pool + 20.6 arena + 1.97 vision = 37.0 -> 1.5 GiB headroom
# VERIFIED at boot: budget 38.5, need 37.0, pool reserved at 524288 with no
# clamp and no WARNING. But the server now holds 103.5 GiB and the ledger
# LOWERED 2026-09-12 from 24 (vendor default is 20) to buy the doubled pool at
# full prefill speed. The fit check's OWN requirement is the line to honour —
# read from a clamped boot, not modelled:
# "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5
# budget = 121.2 host - 67.7 weights - reserve
# so reserve 16 -> 37.5 (1.0 spare, deployed) and 15 -> 38.5 (2.0 spare).
# reserve 18 -> 35.5 CLAMPS the pool back to 262144; 17 fits by 0.02, which is
# inside the noise of the 67.7 measurement. Do not raise past 16.
# Measured at 15 before moving to 16: pool reserved at 524288 with no clamp and
# no WARNING, but the server held 103.5 GiB and the ledger
# reads "host memory left for everything else: 310 contiguous 2 MiB blocks
# (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve
# baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set).
@@ -141,9 +145,10 @@ Environment=HALOGEN_MAX_TOK=32768
# crash and needs no restart.
# That is the failure mode to watch, and it is NOT the legible one predicted
# above: a minutes-long stall that looks like a hang is harder to diagnose
# than a slow prompt. If frank-large or a neighbour ever goes quiet with the
# CPU pinned, raise this reserve or drop the pool back to 262144.
Environment=HALOGEN_HOST_RESERVE_GIB=15
# than a slow prompt. If it happens, the reserve has no headroom left to give
# — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or
# HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24.
Environment=HALOGEN_HOST_RESERVE_GIB=16
# Image input (the sidecar ships inside the weights repo).
Environment=HALOGEN_VISION_TOWER=1
# Verbose engine logging: the vendor's first-line evidence for load,