dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-large: revert reserve to 15 — the knob cannot buy host headroom

Browse Source 
Tested 16 live. Pool stayed 524288 (no clamp), server held 103.5 GiB and
reported "7.1 GiB left for everything else" at BOTH 15 and 16 — identical
footprint. HALOGEN_HOST_RESERVE_GIB feeds the fit-check budget only; it is not
an allocation, so raising it cannot give RAM back to the co-tenants. Its sole
observable effect above 16 is to take the KV pool away.

16 therefore cost half the fit margin (1.0 vs 2.0 against the engine's 36.5 GiB
requirement) and bought nothing. 15 is strictly better at the same cost.

Recorded the ceiling (16) and the real air levers in the unit comment: pool
262144 frees 7.2 GiB, max_tok 16384 frees 8.4.
This commit is contained in:
Frank
2026-09-12 21:30:15 -07:00
parent b87f18569a
commit 5c4e8a9600
1 changed files with 13 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dankclaw/frank-large/quadlet/frank-large.container
+13 -6
View File
@@ -130,11 +130,18 @@ Environment=HALOGEN_MAX_TOK=32768
# read from a clamped boot, not modelled: # read from a clamped boot, not modelled:
# "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5 # "524288 positions need ~35.0 GiB (plus 1.5 of margin)" -> budget >= 36.5
# budget = 121.2 host - 67.7 weights - reserve # budget = 121.2 host - 67.7 weights - reserve
# so reserve 16 -> 37.5 (1.0 spare, deployed) and 15 -> 38.5 (2.0 spare). # reserve 16 -> 37.5 (1.0 spare); 15 -> 38.5 (2.0 spare, deployed); 18 -> 35.5
# reserve 18 -> 35.5 CLAMPS the pool back to 262144; 17 fits by 0.02, which is # CLAMPS the pool to 262144. Do not raise past 16.
# inside the noise of the 67.7 measurement. Do not raise past 16. #
# Measured at 15 before moving to 16: pool reserved at 524288 with no clamp and # CEILING IS 16, AND 16 BUYS NOTHING. Tested live: 15 -> 16 changed the fit
# no WARNING, but the server held 103.5 GiB and the ledger # margin only. The server held 103.5 GiB and reported "7.1 GiB left for
# everything else" at BOTH settings, because this reserve feeds the fit-check
# budget and is not an allocation — actual footprint is weights + KV + arena +
# vision, none of which it touches. A higher reserve therefore cannot give the
# host memory back; it can only take the pool away. 15 keeps the larger margin.
#
# Measured at 15 before that test: pool reserved at 524288 with no clamp and no
# WARNING, and the server held 103.5 GiB. The ledger
# reads "host memory left for everything else: 310 contiguous 2 MiB blocks # reads "host memory left for everything else: 310 contiguous 2 MiB blocks
# (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve # (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve
# baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set). # baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set).
@@ -148,7 +155,7 @@ Environment=HALOGEN_MAX_TOK=32768
# than a slow prompt. If it happens, the reserve has no headroom left to give # than a slow prompt. If it happens, the reserve has no headroom left to give
# — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or # — the remedy is HALOGEN_KV_POOL_POSITIONS=262144 (hands back 7.2 GiB), or
# HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24. # HALOGEN_MAX_TOK=16384 (hands back 8.4) with reserve then free to rise to 24.
Environment=HALOGEN_HOST_RESERVE_GIB=16 Environment=HALOGEN_HOST_RESERVE_GIB=15
# Image input (the sidecar ships inside the weights repo). # Image input (the sidecar ships inside the weights repo).
Environment=HALOGEN_VISION_TOWER=1 Environment=HALOGEN_VISION_TOWER=1
# Verbose engine logging: the vendor's first-line evidence for load, # Verbose engine logging: the vendor's first-line evidence for load,