frank-large: reserve 15 -> 16 for co-tenant headroom
Operator asked for 18. 18 is not available: the engine's own fit-check line,
read from a clamped boot rather than modelled, requires budget >= 36.5 GiB to
keep 524288 at max_tok 32768 ("524288 positions need ~35.0 GiB (plus 1.5 of
margin)"). budget(18) = 35.5, so 18 silently un-doubles the pool we just paid
the reserve for. 17 fits by 0.02, inside the noise of the 67.7 resident-weights
term. 16 is the largest reserve that survives the check with real margin:
37.5 budget vs 36.5 required.
Recorded the ceiling and both remedies in the unit comment. The reserve is now
spent, so a future stall is answered by the pool or the arena, not here.
This commit is contained in:
@@ -125,12 +125,16 @@ Environment=HALOGEN_KV_SLOTS=4
|
||||
# — raise the reserve back to 24 and the pool silently clamps to 262144 again.
|
||||
Environment=HALOGEN_MAX_TOK=32768
|
||||
# RAM left for the n-gram table the engine reads through the page cache.
|
||||
# LOWERED 2026-09-12 from 24 (vendor default is 20) to 15 to buy the doubled
|
||||
# pool at full prefill speed:
|
||||
# 121.2 host - 67.7 weights - 15 reserve = 38.5 budget
|
||||
# need = 14.4 pool + 20.6 arena + 1.97 vision = 37.0 -> 1.5 GiB headroom
|
||||
# VERIFIED at boot: budget 38.5, need 37.0, pool reserved at 524288 with no
|
||||
# clamp and no WARNING. But the server now holds 103.5 GiB and the ledger
|
||||
# LOWERED 2026-09-12 from 24 (vendor default is 20) to buy the doubled pool at
|
||||
# full prefill speed. The fit check's OWN requirement is the line to honour —
|
||||
# read from a clamped boot, not modelled:
|
||||
# "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5
|
||||
# budget = 121.2 host - 67.7 weights - reserve
|
||||
# so reserve 16 -> 37.5 (1.0 spare, deployed) and 15 -> 38.5 (2.0 spare).
|
||||
# reserve 18 -> 35.5 CLAMPS the pool back to 262144; 17 fits by 0.02, which is
|
||||
# inside the noise of the 67.7 measurement. Do not raise past 16.
|
||||
# Measured at 15 before moving to 16: pool reserved at 524288 with no clamp and
|
||||
# no WARNING, but the server held 103.5 GiB and the ledger
|
||||
# reads "host memory left for everything else: 310 contiguous 2 MiB blocks
|
||||
# (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve
|
||||
# baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set).
|
||||
@@ -141,9 +145,10 @@ Environment=HALOGEN_MAX_TOK=32768
|
||||
# crash and needs no restart.
|
||||
# That is the failure mode to watch, and it is NOT the legible one predicted
|
||||
# above: a minutes-long stall that looks like a hang is harder to diagnose
|
||||
# than a slow prompt. If frank-large or a neighbour ever goes quiet with the
|
||||
# CPU pinned, raise this reserve or drop the pool back to 262144.
|
||||
Environment=HALOGEN_HOST_RESERVE_GIB=15
|
||||
# than a slow prompt. If it happens, the reserve has no headroom left to give
|
||||
# — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or
|
||||
# HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24.
|
||||
Environment=HALOGEN_HOST_RESERVE_GIB=16
|
||||
# Image input (the sidecar ships inside the weights repo).
|
||||
Environment=HALOGEN_VISION_TOWER=1
|
||||
# Verbose engine logging: the vendor's first-line evidence for load,
|
||||
|
||||
Reference in New Issue
Block a user