diff --git a/dankclaw/frank-large/quadlet/frank-large.container b/dankclaw/frank-large/quadlet/frank-large.container index f846abc..17ddbfe 100644 --- a/dankclaw/frank-large/quadlet/frank-large.container +++ b/dankclaw/frank-large/quadlet/frank-large.container @@ -130,11 +130,18 @@ Environment=HALOGEN_MAX_TOK=32768 # read from a clamped boot, not modelled: # "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5 # budget = 121.2 host - 67.7 weights - reserve -# so reserve 16 -> 37.5 (1.0 spare, deployed) and 15 -> 38.5 (2.0 spare). -# reserve 18 -> 35.5 CLAMPS the pool back to 262144; 17 fits by 0.02, which is -# inside the noise of the 67.7 measurement. Do not raise past 16. -# Measured at 15 before moving to 16: pool reserved at 524288 with no clamp and -# no WARNING, but the server held 103.5 GiB and the ledger +# reserve 16 -> 37.5 (1.0 spare); 15 -> 38.5 (2.0 spare, deployed); 18 -> 35.5 +# CLAMPS the pool to 262144. Do not raise past 16. +# +# CEILING IS 16, AND 16 BUYS NOTHING. Tested live: 15 -> 16 changed the fit +# margin only. The server held 103.5 GiB and reported "7.1 GiB left for +# everything else" at BOTH settings, because this reserve feeds the fit-check +# budget and is not an allocation — actual footprint is weights + KV + arena + +# vision, none of which it touches. A higher reserve therefore cannot give the +# host memory back; it can only take the pool away. 15 keeps the larger margin. +# +# Measured at 15 before that test: pool reserved at 524288 with no clamp and no +# WARNING, and the server held 103.5 GiB. The ledger # reads "host memory left for everything else: 310 contiguous 2 MiB blocks # (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve # baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set). @@ -148,7 +155,7 @@ Environment=HALOGEN_MAX_TOK=32768 # than a slow prompt. If it happens, the reserve has no headroom left to give # — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or # HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24. -Environment=HALOGEN_HOST_RESERVE_GIB=16 +Environment=HALOGEN_HOST_RESERVE_GIB=15 # Image input (the sidecar ships inside the weights repo). Environment=HALOGEN_VISION_TOWER=1 # Verbose engine logging: the vendor's first-line evidence for load,