frank-large: double the KV pool to 524288 positions

HALOGEN_KV_POOL_POSITIONS 262144 -> 524288, which is upstream's own
default (2 x HALOGEN_CTX). The old pool was the binding constraint on
concurrency, not context: a request reserves prompt + max_tokens against
the pool, so with pool == CTX four concurrent 65k chats were contending
for one full-length conversation's worth of positions. Two full-length
conversations, or four at 131k, now stay resident.

Cost is ~7.2 GiB more device RAM (measured 28 KiB/position), taken from
the page cache the 47.7 GiB n-gram lookup table reads through. Live
startup ledger for the prior config: 68.0 weights + 7.2 pool + 21.1
working = 96.3 GiB held, 12.1 GiB left on the host. The doubled pool
puts the remainder near 5 GiB against a ~4.3 GiB co-tenant working set,
and cache at ~69 GiB -- still above the lookup table, so the table can
stay hot. Compensating lever if long prompts start crawling:
HALOGEN_MAX_TOK=16384, which frees 8.3 GiB of prefill arena for ~8%
of prefill throughput.

Docs updated to match, including the admission-trap paragraph, which was
only true while pool == CTX.

serverAppConfigs-docs/LLM/halogen-flash-server.md

@@ -11,7 +11,7 @@ tags:
   - gfx1151
   - qwen4exp
   - rocm
-timestamp: 2026-09-11
+timestamp: 2026-09-12
 ---
 
 # halogen-flash-server on Dankclaw (frank-large)
@@ -78,7 +78,7 @@ upstream's defaults:
 |---|---|---|
 | `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
 | `HALOGEN_CTX` | `262144` | the model's native context, and what both clients declare |
-| `HALOGEN_KV_POOL_POSITIONS` | `262144` | conservative for a **shared** 121 GiB host: one full-length conversation, or four at 65k, at ~7.2 GiB |
+| `HALOGEN_KV_POOL_POSITIONS` | `524288` | **doubled 2026-09-12** to upstream's own default (2 × `HALOGEN_CTX`): two full-length conversations resident, or four at 131k, at ~14.4 GiB |
 | `HALOGEN_KV_SLOTS` | `4` | four concurrent generations; slots share the pool and cost ~115 MiB each |
 | `HALOGEN_HOST_RESERVE_GIB` | `24` | RAM kept for the n-gram table the engine reads through the page cache — above upstream's 20 because a dozen containers share this host |
 | `HALOGEN_VISION_TOWER` | `1` | image input; the fleet's only image path since [[frank-npu]] was retired |
@@ -103,7 +103,8 @@ Three numbers explain the whole box:
   file cache, so `free`, `MemAvailable` and every dashboard built on them
   **overstate free memory by the size of the model**. The server prints the
   corrected figure at startup; believe that line, not Grafana.
-- **~7.2 GiB per 262,144 KV positions**, plus the prefill arena (16.7 GiB at
+- **~28 KiB per KV position** (measured: 262,144 positions = 6,656 MiB of KV,
+  ~7.2 GiB with block scratch), plus the prefill arena (16.7 GiB at
   `HALOGEN_MAX_TOK=32768`, 8.4 GiB at 16384). The engine's device-side budget
   stops at roughly 47 GiB on a 128 GB machine *with the weights pinned* — and
   lower on a machine carrying more besides this server, which Dankclaw is.
@@ -116,6 +117,23 @@ Three numbers explain the whole box:
 the pool itself if the configured one will not fit, printing what it chose.
 Read those lines on every restart; they are the ground truth for sizing.
 
+**Measured ledger (prior 262,144 config, read live 2026-09-12).** The
+engine's own startup account: 68.0 GiB of weights locked + 7.2 GiB KV pool +
+21.1 GiB working memory = **96.3 GiB held**, with **12.1 GiB left on the
+host**, only ~7.3 GiB of that in contiguous 2 MiB pieces. The 47.7 GiB n-gram
+lookup table is never held in RAM — it is read through the file cache — and
+the 2.0 GiB firmware carves for the iGPU appears in no `/proc/meminfo`
+figure.
+
+Doubling to 524,288 puts the held total near **103.5 GiB** and the host
+remainder at roughly **5 GiB**, against a measured ~4.3 GiB working set for
+the 25 co-tenant containers on this box. The extra 7.2 GiB comes out of the
+page cache, which falls from ~76 GiB to ~69 GiB — still above the 47.7 GiB
+lookup table, so the table can stay hot. Tight, not an overcommit. If long
+prompts start crawling with the disk busy, `HALOGEN_MAX_TOK=16384` shrinks
+the prefill arena 16.7 → 8.4 GiB and pays for the doubling outright at about
+8 % of prefill throughput.
+
 **Co-tenancy.** `HALOGEN_FLASH_PIN_TRUNK=1` (default) is what the published
 decode numbers were measured with, and it means this container holds most of
 the host. If a co-tenant then takes the remainder, both it and the engine can
@@ -156,9 +174,12 @@ deployment:
 **plus** `max_tokens` positions when it is admitted; if the pool cannot hold it
 the request waits in arrival order, and after the queue timeout (3600 s) it
 gets a `503 engine_busy`. A prompt large enough that prompt + budget exceeds
-the pool can therefore never be admitted at all. The 262,144-position pool is
-the binding constraint on how large a conversation can get, not `HALOGEN_CTX`
-and not the client's declared `context_length`.
+the pool can therefore never be admitted at all. With the pool at 524,288 the
+per-conversation ceiling is `HALOGEN_CTX` (262,144) — the pool binds **how
+many** conversations stay resident, not how long any one of them can be. Under
+the previous 262,144 pool the two numbers were identical, which is why four
+concurrent 65k chats were contending for one conversation's worth of
+positions.
 
 ## Vision