diff --git a/AGENTS.md b/AGENTS.md index 0c1cbaf..02873d4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -419,16 +419,17 @@ disagrees with it is the bug. digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`. Engine stdout (the boot ledger: weights, KV pool, clamp) goes to `/home/dank/logs/frank_dragoon.log`, NOT the journal. -- Shape: `HALOGEN_CTX=200704` is the **per-request ceiling** (a longer prompt - is a 400, never a truncation); `HALOGEN_KV_SLOTS=2` over ONE - `HALOGEN_KV_POOL_POSITIONS=401408` pool — pool positions buy concurrency, - the per-request window never moves; `HALOGEN_SPEC_ADAPT=0` keeps the MTP - drafter always armed (the adaptive default disarms it and halves decode); - server sampler 1.0/0.95/20. -- Harness contract: `models.yml` `contextWindow: 155648` = ceiling − 45,056. - The clamp is `min(45056, window − est − 4096)` and the chars/4 estimate - under-counts, so a harness window equal to the ceiling makes every long - request a guaranteed 400. Reasoning arrives in `reasoning_content`; the +- Shape: `HALOGEN_CTX=262144` (the checkpoint's native window) is the + **per-request ceiling** (a longer prompt is a 400, never a truncation); + `HALOGEN_KV_SLOTS=2` over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool — + pool positions buy concurrency, the per-request window never moves; + `HALOGEN_SPEC_ADAPT=0` keeps the MTP drafter always armed (the adaptive + default disarms it and halves decode); server sampler 1.0/0.95/20. +- Harness contract: `models.yml` `contextWindow: 200000` — ~45k+ under the + ceiling. The clamp is `min(45056, window − est − 4096)` and the chars/4 + estimate under-counts (trigger is 95 %), so a harness window at the ceiling + makes every long request a guaranteed 400. + Reasoning arrives in `reasoning_content`; the template rejects `reasoning_effort` outside `xhigh|medium|low`; images are OpenAI `image_url` parts (probe ≥256 px); `response_format` unsupported; `/completion`, `/ping`, `/props`, `/tokenize` are 404. diff --git a/serverAppConfigs-docs/Agent/OMP Harness.md b/serverAppConfigs-docs/Agent/OMP Harness.md index 239e01b..1c5a80e 100644 --- a/serverAppConfigs-docs/Agent/OMP Harness.md +++ b/serverAppConfigs-docs/Agent/OMP Harness.md @@ -52,28 +52,31 @@ Where the contract lives: The arithmetic (the coupled contract every endpoint's maxTokens comment encodes). The authoritative `contextWindow`/`maxTokens` pair per model is on the endpoint page ([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]); what -follows is a **worked example** recomputed from the live files on 2026-10-18 — -a snapshot, not fleet state — with windows 200,000 / 155,648 / 200,704 and +follows is a **worked example** recomputed from the live files on 2026-10-21 — +a snapshot, not fleet state — with windows 200,000 / 200,000 / 200,704 and maxTokens 16,384 / 45,056 / 16,384 (archon / dragoon / zealot): 1. **Source trigger** = `thresholdPercent × source contextWindow`; at the live - 95 % that is 190,000 (archon), 147,866 (dragoon), 190,669 (zealot). The + 95 % that is 190,000 (archon and dragoon), 190,669 (zealot). The bundle clamps the percentage to 1–99 and converts it to a token threshold. 2. **Output budget** = capped at the target model's `maxTokens` — every role here compacts to frank-zealot, so the budget is its 16,384. -3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot - window. **At 95 % this no longer holds**: 190,000 + 16,384 = 206,384 > - 200,704. The registry's own comments design around an 85 % trigger (archon: - 170,000 est × 1.03 + 16,384 ≈ 191,500 ≤ 200,704, `models.yml:49-56`), which - does hold. While `config.yml` and those comments disagree, treat 85 % as the - design point and re-run this arithmetic after ANY window change — a 400 from - the compaction target mid-conversation is the symptom. +3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the fitting window. For a + dragoon session the fitting window is the ENGINE ceiling, not a slot: the + 190,000 trigger is 62,144 under the 262,144 ceiling, so even ×1.03 est + error plus the 45,056 request budget lands inside — that headroom is the + reason the ceiling moved to native on 2026-10-21. For a session compacting + INTO frank-zealot the window is its 200,704-token slot, and **at 95 % this + does not hold for archon/zealot sources**: 190,000 + 16,384 = 206,384 > + 200,704. Treat 85 % as the design point until config.yml and the registry + comments agree, and re-run this arithmetic after ANY window change — a 400 + from the compaction target mid-conversation is the symptom. (Historic 2026-10-05 pairs, at `thresholdPercent: 70`: 140,492 + 36,864 = 177,356 ≤ 200,704 archon; 140,490 + 57,344 ≤ 200,700 dragoon; 140,492 + 32,768 ≤ 200,704 zealot.) Procedure (recompute from the LIVE files every time — windows rotate with -every swap; the block above is the 2026-10-18 state): +every swap; the block above is the 2026-10-21 state): 1. Read the `config.yml` compaction block + `models.yml` entries for source and target. Every endpoint's `contextWindow` comment carries its own @@ -94,9 +97,10 @@ Known reconciliation trap: comments in models.yml have described compaction as running "on the dankserver endpoint" while `remoteEndpoint` pointed at dankclaw — the comment/config disagreed. The comments now name the real target (`compaction.remoteEndpoint` = dankclaw :7771). **The config is -truth; fix comments in the same change.** A fresh instance of the same trap is -open 2026-10-18: every `maxTokens` comment still says "compaction fires at -85 % of the window" while `config.yml:84` sets `thresholdPercent: 95`. +truth; fix comments in the same change.** The 85 %-comments vs 95 %-config +disagreement is not academic: it is why long dragoon sessions kept drawing +400s at the old 200,704 ceiling — the trigger fires late AND the est +under-counts, and a window parked near the ceiling has no room for either. `maxTokens` never changes for a reslice — it caps output; `contextWindow` caps input. Boot-shape OOM laddering is a different problem ([[Llama Cpp Serving Sizing]]). diff --git a/serverAppConfigs-docs/LLM/frank-dragoon.md b/serverAppConfigs-docs/LLM/frank-dragoon.md index 8765b93..a0077fc 100644 --- a/serverAppConfigs-docs/LLM/frank-dragoon.md +++ b/serverAppConfigs-docs/LLM/frank-dragoon.md @@ -10,14 +10,15 @@ tags: - gfx1151 - halogen - endpoint -timestamp: 2026-10-19 +timestamp: 2026-10-21 --- # frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU) Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18 -(`2b12716`). This page replaces every per-era dragoon runbook; the Eras table is the rollback record. +(`2b12716`), ceiling raised to the native 262,144 on 2026-10-21 so the harness window is 200,000. +This page replaces every per-era dragoon runbook; the Eras table is the rollback record. ## Current shape @@ -37,10 +38,12 @@ Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b unauthenticated token port 8730 stays on container loopback. Engine stdout → `/home/dank/logs/frank_dragoon.log`, **not** the journal. -- Shape: `HALOGEN_CTX=200704` hard per-request ceiling · `HALOGEN_KV_SLOTS=2` · - `HALOGEN_KV_POOL_POSITIONS=401408` = slots × ctx, the top of the band `tests/frank_dragoon_unit.bats` - enforces (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob - exists here, the pool is the only KV lever · native window 262,144, `HALOGEN_ROPE_YARN` beyond it. +- Shape: `HALOGEN_CTX=262144` hard per-request ceiling = the checkpoint's native window (raised + 2026-10-21 from 200704; `HALOGEN_ROPE_YARN` beyond native is not on plan) · `HALOGEN_KV_SLOTS=2` · + `HALOGEN_KV_POOL_POSITIONS=401408` inside the band `tests/frank_dragoon_unit.bats` enforces + (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob + exists here, the pool is the only KV lever · the pool carries TWO maximum harness-length (200k) + conversations at once; two literal ceiling-length requests would queue — that is a pool raise. - Knobs: `HALOGEN_MAX_TOK=8192` sizes the prefill arena (5.7 GiB) and is only the default for requests omitting `max_tokens` · `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair · MTP depth 2 with the head INSIDE the checkpoint (**`HALOGEN_MTP_HEAD` stays unset**) and @@ -53,10 +56,11 @@ unauthenticated token port 8730 stays on container loopback. Engine stdout → (Pin::None)`; no clamp line. Host `MemAvailable` read 22 GiB right after the 2026-10-18 raise — ~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observability). -- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648` - (= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]` - `reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction - rides [[frank-zealot]]. +- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 200000` + — sits 62,144 under the ceiling, clearing the ~45k fit-law gap (400-zone law below); + `tests/frank_dragoon_unit.bats` fails if the page's declared window and `HALOGEN_CTX` drift back + together. `maxTokens: 45056`, `input: [text, image]`, `reasoningContentField: "reasoning_content"`, + `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction rides [[frank-zealot]]. - Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights, pool, MTP, prompt cache, **no clamp**) + `/health` 200 + a tool call + the 512 px red-PNG probe. Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it) and `amdgpu.noretry=0` — the env says check it at @@ -124,14 +128,18 @@ created it — earlier topologies in . prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in 85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so re-measure the warm ratio after any pool change. -- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow: - 155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096` - its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The - 3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 — - over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k - target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request - is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 % - under-count. Compaction rides [[frank-zealot]] at ≈ threshold − keepRecent(20k) ≈ 112k input. +- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling, because the harness + clamps `max_tokens` to `contextWindow − est − 4096`, its chars/4 `est` UNDER-counts code/agent + traffic by ~3 %, and the compaction trigger fires at 95 % of the window (`config.yml + thresholdPercent`, while the comments designed around 85 % — check the trigger BEFORE touching a + window; that disagreement is why long sessions kept drawing 400s). The 3 % is measured, not + theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 — over by 354, a 400. + At the live pair (ceiling 262,144 / window 200,000) the worst-case ordinary request is + 200,000 − 4,096 + 3 % ≈ 201.8k, and the trigger point is 190,000 est × 1.03 + 45,056 ≈ 240.7k — + both inside the ceiling with ~60k of margin. Compaction rides [[frank-zealot]] at + ≈ threshold − keepRecent(20k) ≈ 175k measured against its 200,704-token slot. The seam in + `tests/frank_dragoon_unit.bats` re-runs this arithmetic at the 95 % trigger and reads the window + from THIS page, so ceiling/window pairs that drift back together fail the suite, not a session. - **Admission reserves prompt + `max_tokens`:** an un-fittable request waits in arrival order, then 503s `engine_busy` for the whole queue after `QUEUE_TIMEOUT` — and the cap/timeout pair moves together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).