diff --git a/AGENTS.md b/AGENTS.md index 02aec46..63dfd5c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -32,7 +32,7 @@ host. | Target | Runtime | Service | Model | Port | |--------|---------|---------|-------|------| | Dankserver | FreeToken v0.1.2 | frank-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 | -|Dankclaw|halogen-flash-server 0.5.8 (rootless Quadlet container)|frank-large|peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn 115.55 GiB + quality overlay, ~68 GiB pinned, 262144 ctx / 4 slots, pool 524288 / max_tok 32768 / reserve 15, vision tower on, greedy for silent clients)|7770| +|Dankclaw|halogen-flash-server 0.5.8 (rootless Quadlet container)|frank-large|peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn 115.55 GiB + quality overlay, ~68 GiB pinned, 262144 ctx / 3 slots, pool 524288 / max_tok 32768 / reserve 15, vision tower on, greedy for silent clients)|7770| | Dankclaw | *(retired 2026-09-11)* FastFlowLM v1.0.1 on the XDNA2 NPU | frank-npu | Gemma4-E4B-IT-NPU2 pack (vision + audio). Retired with the `amd_iommu=off` decision: the driver's SVA/PASID bind needs an enabled IOMMU | 7771 | History: the 2026-08-17 redo retired the pipeline-parallel split and the @@ -397,7 +397,8 @@ with `HALOGEN_MODEL_ID=frank-large`, so no client wiring changed. Deployed flags: `HALOGEN_CTX=262144`, `HALOGEN_KV_POOL_POSITIONS=524288` (**in force** since 2026-09-12 — two full-length conversations resident, or -four at 131k), `HALOGEN_KV_SLOTS=4`, `HALOGEN_MAX_TOK=32768`, +four at 131k), `HALOGEN_KV_SLOTS=3` (lowered from 4 on 2026-09-12 to the real +fleet size; 111 MiB of O(1) state each), `HALOGEN_MAX_TOK=32768`, `HALOGEN_HOST_RESERVE_GIB=15`, `HALOGEN_VISION_TOWER=1`, and `HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next` for the first-start fetch; everything else is upstream's default. **Pool and reserve are a matched diff --git a/serverAppConfigs-docs/LLM/halogen-flash-server.md b/serverAppConfigs-docs/LLM/halogen-flash-server.md index 4808c7c..4c28c71 100644 --- a/serverAppConfigs-docs/LLM/halogen-flash-server.md +++ b/serverAppConfigs-docs/LLM/halogen-flash-server.md @@ -79,7 +79,7 @@ upstream's defaults: | `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename | | `HALOGEN_CTX` | `262144` | the model's native context, and what both clients declare | | `HALOGEN_KV_POOL_POSITIONS` | `524288` | **in force** — two full-length conversations resident, or four at 131k, 13,312 MiB of KV. Was silently clamped to 262,144 until the reserve below dropped; `HALOGEN_KV_POOL_FIT` still guards the boot, so read the ledger rather than trusting this column | -| `HALOGEN_KV_SLOTS` | `4` | four concurrent generations; slots share the pool and cost ~115 MiB each | +| `HALOGEN_KV_SLOTS` | `3` | Concurrency cap — **lowered from 4 on 2026-09-12** to the real fleet size. Each slot is 111.0 MiB of O(1) state, so this freed ~111 MiB (held 103.5 → 103.4 GiB). Slots share the ONE pool and **oversubscribe** it (4 × 262144 > 524288), so admission was always pool-bound: at most two full-length conversations, or four at 131k. This trims simultaneous requests, not how much context fits | | `HALOGEN_MAX_TOK` | `32768` | upstream's default, kept. The arena is 20.6 GiB here against 12.2 at 16384; it only fits alongside the doubled pool because the reserve dropped to 15. At 65536 the arena alone exceeds the budget and the server **will not boot** | | `HALOGEN_HOST_RESERVE_GIB` | `15` | **lowered from 24 (vendor default 20) on 2026-09-12** to buy both features. The only term you own in the device budget (24 → 29.5 GiB, 15 → 38.5 GiB), and **the ceiling for the doubled pool at `max_tok=32768` is 16** — the engine needs budget ≥ 36.5, and 18 gives 35.5, which silently re-clamps the pool. Matched pair with the pool. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away | | `HALOGEN_VISION_TOWER` | `1` | image input; the fleet's only image path since [[frank-npu]] was retired. Charges 1.97 GiB against the budget — the term missing from my first estimate |