frank-large: document reserve 15 result — both features in force, stall risk logged
Ledger confirms the doubled pool is actually reserved (524288, 13312.0 MiB) at max_tok 32768 with no clamp and no WARNING: 38.5 budget vs 37.0 needed. Recorded the cost honestly. The server now holds 103.5 of 121.2 GiB and the engine reports only 310 contiguous 2 MiB blocks / 7.1 GiB left for everything else (24-reserve baseline: 2081 blocks / 12.1 GiB). All 28 co-tenants stayed up, but the engine volunteers a new note: when memory runs out, frank-large AND its neighbours "can stall for minutes at a time at full CPU with no disk activity and no output, which is not a crash and needs no restart". That is the least diagnosable of the three failure modes seen today, and `free` hides it — MemAvailable still reads ~74 GiB because the pinned weights count as reclaimable page cache. Sizing rule now expresses the budget as a function of reserve, shows both endpoints (24 -> 29.5, 15 -> 38.5), and the max_tok ceiling is labelled as reserve-conditional (<= 18,130 at 24; <= 35,700 at 15).
This commit is contained in:
@@ -129,11 +129,20 @@ Environment=HALOGEN_MAX_TOK=32768
|
||||
# pool at full prefill speed:
|
||||
# 121.2 host - 67.7 weights - 15 reserve = 38.5 budget
|
||||
# need = 14.4 pool + 20.6 arena + 1.97 vision = 37.0 -> 1.5 GiB headroom
|
||||
# Thin on purpose; do not go below 15 without re-doing that arithmetic. The
|
||||
# failure mode here is legible rather than fatal — long prompts fault heavily
|
||||
# against a busy disk, and the remedy is to raise this number until the pool
|
||||
# hands the memory back. Note the reserve was never held literally: at the
|
||||
# previous 24 setting the engine's own ledger reported only 12-17 GiB left.
|
||||
# VERIFIED at boot: budget 38.5, need 37.0, pool reserved at 524288 with no
|
||||
# clamp and no WARNING. But the server now holds 103.5 GiB and the ledger
|
||||
# reads "host memory left for everything else: 310 contiguous 2 MiB blocks
|
||||
# (7.1 GiB total)" — against 2081 blocks / 12.1 GiB at the 24-reserve
|
||||
# baseline. All 28 co-tenant containers stayed up (~4.3 GiB working set).
|
||||
#
|
||||
# The price is that the engine volunteers this note on every boot now:
|
||||
# when it runs out, BOTH they and this server can stall for minutes at a
|
||||
# time at full CPU with no disk activity and no output, which is not a
|
||||
# crash and needs no restart.
|
||||
# That is the failure mode to watch, and it is NOT the legible one predicted
|
||||
# above: a minutes-long stall that looks like a hang is harder to diagnose
|
||||
# than a slow prompt. If frank-large or a neighbour ever goes quiet with the
|
||||
# CPU pinned, raise this reserve or drop the pool back to 262144.
|
||||
Environment=HALOGEN_HOST_RESERVE_GIB=15
|
||||
# Image input (the sidecar ships inside the weights repo).
|
||||
Environment=HALOGEN_VISION_TOWER=1
|
||||
|
||||
Reference in New Issue
Block a user