diff --git a/dankclaw/frank-large/quadlet/frank-large.container b/dankclaw/frank-large/quadlet/frank-large.container index 6e90554..f846abc 100644 --- a/dankclaw/frank-large/quadlet/frank-large.container +++ b/dankclaw/frank-large/quadlet/frank-large.container @@ -125,12 +125,16 @@ Environment=HALOGEN_KV_SLOTS=4 # — raise the reserve back to 24 and the pool silently clamps to 262144 again. Environment=HALOGEN_MAX_TOK=32768 # RAM left for the n-gram table the engine reads through the page cache. -# LOWERED 2026-09-12 from 24 (vendor default is 20) to 15 to buy the doubled -# pool at full prefill speed: -# 121.2 host - 67.7 weights - 15 reserve = 38.5 budget -# need = 14.4 pool + 20.6 arena + 1.97 vision = 37.0 -> 1.5 GiB headroom -# VERIFIED at boot: budget 38.5, need 37.0, pool reserved at 524288 with no -# clamp and no WARNING. But the server now holds 103.5 GiB and the ledger +# LOWERED 2026-09-12 from 24 (vendor default is 20) to buy the doubled pool at +# full prefill speed. The fit check's OWN requirement is the line to honour — +# read from a clamped boot, not modelled: +# "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5 +# budget = 121.2 host - 67.7 weights - reserve +# so reserve 16 -> 37.5 (1.0 spare, deployed) and 15 -> 38.5 (2.0 spare). +# reserve 18 -> 35.5 CLAMPS the pool back to 262144; 17 fits by 0.02, which is +# inside the noise of the 67.7 measurement. Do not raise past 16. +# Measured at 15 before moving to 16: pool reserved at 524288 with no clamp and +# no WARNING, but the server held 103.5 GiB and the ledger # reads "host memory left for everything else: 310 contiguous 2 MiB blocks # (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve # baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set). @@ -141,9 +145,10 @@ Environment=HALOGEN_MAX_TOK=32768 # crash and needs no restart. # That is the failure mode to watch, and it is NOT the legible one predicted # above: a minutes-long stall that looks like a hang is harder to diagnose -# than a slow prompt. If frank-large or a neighbour ever goes quiet with the -# CPU pinned, raise this reserve or drop the pool back to 262144. -Environment=HALOGEN_HOST_RESERVE_GIB=15 +# than a slow prompt. If it happens, the reserve has no headroom left to give +# — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or +# HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24. +Environment=HALOGEN_HOST_RESERVE_GIB=16 # Image input (the sidecar ships inside the weights repo). Environment=HALOGEN_VISION_TOWER=1 # Verbose engine logging: the vendor's first-line evidence for load,