dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

frank-large: correct docs to measured 401408 ledger figures

Browse Source 
Predicted ~10.8 GiB KV / ~33.4 need / ~10.5 host-left; the boot ledger
printed ~11.0 / ~31.5 / 100.0 GiB held with 9.1 GiB + 1396 contiguous
2 MiB blocks left. Stall NOTE no longer prints at this footprint. Also
documents the ~45-60 s first-request PING silence after a restart (the
healthcheck's HealthStartPeriod covers it; do not restart the unit).
This commit is contained in:
Frank
2026-09-13 23:02:16 -07:00
parent 91406af2a1
commit 27e63359d6
3 changed files with 33 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/halogen-flash-server.md
+10 -6
View File
@@ -78,7 +78,7 @@ upstream's defaults:
|---|---|---|
| `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
| `HALOGEN_CTX` | `262144` | the model's native context, and what both clients declare |
| `HALOGEN_KV_POOL_POSITIONS` | `401408` | **trimmed 2026-09-13** from the 524,288 the doubling deployed: one full 262k conversation resident beside a ~139k one, or three at 131k, ~10.8 GiB of KV. The trim hands ~3.4 GiB back to the ~28 co-tenants — the doubled pool left only ~7.1 GiB on the host and the engine warned of co-tenant stalls on every boot. `HALOGEN_KV_POOL_FIT` still guards the boot, so read the ledger rather than trusting this column |
| `HALOGEN_KV_POOL_POSITIONS` | `401408` | **trimmed 2026-09-13** from the 524,288 the doubling deployed: one full 262k conversation resident beside a ~139k one, or three at 131k, ~11.0 GiB of KV as reserved. Measured against the doubled pool the host went from 7.1 GiB / 310 contiguous 2 MiB blocks to 9.1 / 1396, and the co-tenant stall NOTE no longer prints. `HALOGEN_KV_POOL_FIT` still guards the boot, so read the ledger rather than trusting this column |
| `HALOGEN_KV_SLOTS` | `3` | Concurrency cap — **lowered from 4 on 2026-09-12** to the real fleet size. Each slot is 111.0 MiB of O(1) state, so this freed ~111 MiB (held 103.5 → 103.4 GiB). Slots share the ONE pool and **oversubscribe** it (3 × 262144 > 401408), so admission is pool-bound: one full-length conversation plus a ~139k second, or three at 131k. This trims simultaneous requests, not how much context fits |
| `HALOGEN_MAX_TOK` | `32768` | upstream's default, kept. The arena is 20.6 GiB here against 12.2 at 16384; it fit alongside the doubled pool only because the reserve dropped to 15, and it fits the trimmed 401,408 pool with room to spare. At 65536 the arena alone exceeds the budget and the server **will not boot** |
| `HALOGEN_HOST_RESERVE_GIB` | `15` | **lowered from 24 (vendor default 20) on 2026-09-12** to buy the doubled pool. The only term you own in the device budget (24 → 29.5 GiB, 15 → 38.5 GiB). Against the doubled pool the ceiling was 16 — the engine needed budget ≥ 36.5, and 18 gave 35.5, silently re-clamping; at 401,408 the need is ~33.4 so the ceiling loosens, but a bigger reserve still buys nothing. Matched pair with the pool. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away |
@@ -175,17 +175,19 @@ fits unmodified.
### What the reserve actually costs
The device budget is not the whole story. Memory position by configuration
(the 09-13 trim added its row after the deploy was measured — see below):
(401,408 is the 2026-09-13 deployed row, read from its boot ledger):
| config | held | host left | contiguous 2 MiB blocks |
|---|---|---|---|
| 262144 / 32768, reserve 24 | 96.3 | 12.1 GiB | 2081 |
| 524288 / 16384, reserve 24 | 94.6 | 13.8 GiB | 3742 |
| **524288 / 32768, reserve 15** | **103.5** | **7.1 GiB** | **310** |
| 524288 / 32768, reserve **16** | **103.5** | **7.1 GiB** | 293 |
| **401408 / 32768, reserve 15** | **100.0** | **9.1 GiB** | **1396** |
At the doubled setting (2026-09-12) the server held 103.5 of 121 GiB and the
engine volunteered a note it had not printed before:
engine volunteered a note it had not printed before (it no longer prints at
the 401,408 trim — 9.1 GiB / 1396 blocks sits below whatever threshold
triggers it):
```
halogen: NOTE this server now holds most of the machine, and 310 contiguous
@@ -230,8 +232,10 @@ thing a bigger reserve can do is take the doubling away.
then becomes free to rise: `HALOGEN_KV_POOL_POSITIONS=262144` hands back
7.2 GiB from the doubled pool, `HALOGEN_MAX_TOK=16384` hands back 8.4. Both
are one-line edits that the fit check will confirm on the next boot. The
2026-09-13 trim to 401,408 was this remedy applied at half strength: ~3.4 GiB
back with `HALOGEN_MAX_TOK` untouched.
2026-09-13 trim to 401,408 was this remedy applied at half strength —
`HALOGEN_MAX_TOK` untouched, and on the host-left metric it bought +2.0 GiB
with a 4.5x improvement in contiguous blocks (310 → 1396), less than the
modelled 3.4 because the engine holds the freed KV as working slack.
### The `WARNING` line only appears when the fit check had to move