frank-large: document reserve 15 result — both features in force, stall risk logged

Ledger confirms the doubled pool is actually reserved (524288, 13312.0 MiB)
at max_tok 32768 with no clamp and no WARNING: 38.5 budget vs 37.0 needed.

Recorded the cost honestly. The server now holds 103.5 of 121.2 GiB and the
engine reports only 310 contiguous 2 MiB blocks / 7.1 GiB left for everything
else (24-reserve baseline: 2081 blocks / 12.1 GiB). All 28 co-tenants stayed
up, but the engine volunteers a new note: when memory runs out, frank-large
AND its neighbours "can stall for minutes at a time at full CPU with no disk
activity and no output, which is not a crash and needs no restart". That is
the least diagnosable of the three failure modes seen today, and `free` hides
it — MemAvailable still reads ~74 GiB because the pinned weights count as
reclaimable page cache.

Sizing rule now expresses the budget as a function of reserve, shows both
endpoints (24 -> 29.5, 15 -> 38.5), and the max_tok ceiling is labelled as
reserve-conditional (<= 18,130 at 24; <= 35,700 at 15).
This commit is contained in:
Frank
2026-09-12 21:22:36 -07:00
parent 8a79f9a709
commit 2969e08616
3 changed files with 93 additions and 31 deletions
@@ -129,11 +129,20 @@ Environment=HALOGEN_MAX_TOK=32768
# pool at full prefill speed:
# 121.2 host - 67.7 weights - 15 reserve = 38.5 budget
# need = 14.4 pool + 20.6 arena + 1.97 vision = 37.0 -> 1.5 GiB headroom
# Thin on purpose; do not go below 15 without re-doing that arithmetic. The
# failure mode here is legible rather than fatal — long prompts fault heavily
# against a busy disk, and the remedy is to raise this number until the pool
# hands the memory back. Note the reserve was never held literally: at the
# previous 24 setting the engine's own ledger reported only 12-17 GiB left.
# VERIFIED at boot: budget 38.5, need 37.0, pool reserved at 524288 with no
# clamp and no WARNING. But the server now holds 103.5 GiB and the ledger
# reads "host memory left for everything else: 310 contiguous 2 MiB blocks
# (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve
# baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set).
#
# The price is that the engine volunteers this note on every boot now:
# when it runs out, BOTH they and this server can stall for minutes at a
# time at full CPU with no disk activity and no output, which is not a
# crash and needs no restart.
# That is the failure mode to watch, and it is NOT the legible one predicted
# above: a minutes-long stall that looks like a hang is harder to diagnose
# than a slow prompt. If frank-large or a neighbour ever goes quiet with the
# CPU pinned, raise this reserve or drop the pool back to 262144.
Environment=HALOGEN_HOST_RESERVE_GIB=15
# Image input (the sidecar ships inside the weights repo).
Environment=HALOGEN_VISION_TOWER=1