docs(halogen): record the reserve ceiling and that the reserve is not an allocation
Seven restarts measured. New section "The reserve cannot buy air": reserve 16 was deployed and held exactly what 15 holds — 103.5 GiB, 7.1 GiB left to the host. The flag is a term in the fit-check inequality, not a reservation, so it has only two regimes: below the pool's requirement it changes nothing observable, above it the pool silently disappears. Sizing rule now defers to the engine's printed requirement (budget >= 36.5 for 524288 at max_tok 32768, from "need ~35.0 GiB plus 1.5 of margin") over my modelled 37.0, and the ledger table carries the reserve-16 row.
This commit is contained in:
@@ -403,8 +403,13 @@ four at 131k), `HALOGEN_KV_SLOTS=4`, `HALOGEN_MAX_TOK=32768`,
|
||||
fetch; everything else is upstream's default. **Pool and reserve are a matched
|
||||
pair:** the budget is `121.2 host − 67.7 weights − reserve`, and that one
|
||||
budget carries the KV pool, the prefill arena and the 1.97 GiB vision tower
|
||||
together. Raise the reserve back to 24 and `HALOGEN_KV_POOL_FIT` silently
|
||||
clamps the pool to 262144 while the deploy still reports healthy. At 15 the
|
||||
together. Raise the reserve past **16** and `HALOGEN_KV_POOL_FIT` silently
|
||||
clamps the pool to 262144 while the deploy still reports healthy (the engine
|
||||
needs budget ≥ 36.5; reserve 18 yields 35.5). The reserve is **not an
|
||||
allocation** — 15 and 16 were both measured holding 103.5 GiB with 7.1 GiB left
|
||||
on the host, so a bigger reserve cannot free memory for co-tenants, it can only
|
||||
remove the pool. To free host RAM, cut `HALOGEN_KV_POOL_POSITIONS` (262144
|
||||
hands back 7.2 GiB) or `HALOGEN_MAX_TOK` (16384 hands back 8.4). At 15 the
|
||||
server holds 103.5 of 121 GiB, leaving ~7.1 GiB on the host, and the engine
|
||||
warns that when it runs out frank-large and its neighbours can stall for
|
||||
minutes at full CPU with no disk activity — not a crash, and invisible to
|
||||
|
||||
Reference in New Issue
Block a user