dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(frank-large): resync slots=3 across AGENTS and runbook

Browse Source 
Ledger confirms the change landed: "3 slot(s) over one 524288-position pool",
"slots: 1 -> 3 POOLED, 111.0 MiB each", held 103.5 -> 103.4 GiB.

Recorded the oversubscription nuance that the old row missed: the slots were
never the binding constraint on context. 4 x 262144 exceeds the 524288 pool,
so admission was pool-bound all along -- two full-length conversations, or
four at 131k. Lowering the cap trims simultaneous requests, not capacity.
This commit is contained in:
Frank
2026-09-12 23:26:03 -07:00
parent 6169278911
commit 4e3d79388b
2 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/halogen-flash-server.md
+1 -1
View File
@@ -79,7 +79,7 @@ upstream's defaults:
| `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
| `HALOGEN_CTX` | `262144` | the model's native context, and what both clients declare |
| `HALOGEN_KV_POOL_POSITIONS` | `524288` | **in force** — two full-length conversations resident, or four at 131k, 13,312 MiB of KV. Was silently clamped to 262,144 until the reserve below dropped; `HALOGEN_KV_POOL_FIT` still guards the boot, so read the ledger rather than trusting this column |
| `HALOGEN_KV_SLOTS` | `4` | four concurrent generations; slots share the pool and cost ~115 MiB each |
| `HALOGEN_KV_SLOTS` | `3` | Concurrency cap — **lowered from 4 on 2026-09-12** to the real fleet size. Each slot is 111.0 MiB of O(1) state, so this freed ~111 MiB (held 103.5 → 103.4 GiB). Slots share the ONE pool and **oversubscribe** it (4 × 262144 > 524288), so admission was always pool-bound: at most two full-length conversations, or four at 131k. This trims simultaneous requests, not how much context fits |
| `HALOGEN_MAX_TOK` | `32768` | upstream's default, kept. The arena is 20.6 GiB here against 12.2 at 16384; it only fits alongside the doubled pool because the reserve dropped to 15. At 65536 the arena alone exceeds the budget and the server **will not boot** |
| `HALOGEN_HOST_RESERVE_GIB` | `15` | **lowered from 24 (vendor default 20) on 2026-09-12** to buy both features. The only term you own in the device budget (24 → 29.5 GiB, 15 → 38.5 GiB), and **the ceiling for the doubled pool at `max_tok=32768` is 16** — the engine needs budget ≥ 36.5, and 18 gives 35.5, which silently re-clamps the pool. Matched pair with the pool. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away |
| `HALOGEN_VISION_TOWER` | `1` | image input; the fleet's only image path since [[frank-npu]] was retired. Charges 1.97 GiB against the budget — the term missing from my first estimate |