feat(frank-dragoon): 200,000-token harness window via the native 262,144 ceiling #65

Merged
dank merged 2 commits from fix/dragoon-context-200k into main 2026-10-21 01:40:40 -07:00
7 changed files with 106 additions and 78 deletions
+11 -10
View File
@@ -419,16 +419,17 @@ disagrees with it is the bug.
digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`.
Engine stdout (the boot ledger: weights, KV pool, clamp) goes to
`/home/dank/logs/frank_dragoon.log`, NOT the journal.
- Shape: `HALOGEN_CTX=200704` is the **per-request ceiling** (a longer prompt
is a 400, never a truncation); `HALOGEN_KV_SLOTS=2` over ONE
`HALOGEN_KV_POOL_POSITIONS=401408` pool — pool positions buy concurrency,
the per-request window never moves; `HALOGEN_SPEC_ADAPT=0` keeps the MTP
drafter always armed (the adaptive default disarms it and halves decode);
server sampler 1.0/0.95/20.
- Harness contract: `models.yml` `contextWindow: 155648` = ceiling − 45,056.
The clamp is `min(45056, window − est − 4096)` and the chars/4 estimate
under-counts, so a harness window equal to the ceiling makes every long
request a guaranteed 400. Reasoning arrives in `reasoning_content`; the
- Shape: `HALOGEN_CTX=262144` (the checkpoint's native window) is the
**per-request ceiling** (a longer prompt is a 400, never a truncation);
`HALOGEN_KV_SLOTS=2` over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool —
pool positions buy concurrency, the per-request window never moves;
`HALOGEN_SPEC_ADAPT=0` keeps the MTP drafter always armed (the adaptive
default disarms it and halves decode); server sampler 1.0/0.95/20.
- Harness contract: `models.yml` `contextWindow: 200000` — ~45k+ under the
ceiling. The clamp is `min(45056, window − est − 4096)` and the chars/4
estimate under-counts (trigger is 95 %), so a harness window at the ceiling
makes every long request a guaranteed 400.
Reasoning arrives in `reasoning_content`; the
template rejects `reasoning_effort` outside `xhigh|medium|low`; images are
OpenAI `image_url` parts (probe ≥256 px); `response_format` unsupported;
`/completion`, `/ping`, `/props`, `/tokenize` are 404.
+4 -3
View File
@@ -1401,9 +1401,10 @@ deploy_anubis() {
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next —
# qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident)
# + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode
# (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 401,408-position KV pool (2 × ctx,
# raised 2026-10-18 so BOTH slots can hold a maximum-length request) at the
# ctx 200704 ceiling, image input — on the Strix Halo iGPU. It stays a
# (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 401,408-position KV pool at the
# native ctx 262144 ceiling (raised 2026-10-21 from 200704 so the harness can
# run a 200,000 window with its ~45k fit gap; the pool still carries two
# 200k conversations at once), image input — on the Strix Halo iGPU. It stays a
# ROOT unit because the serve script drives `podman run` directly (the frank-archon pattern):
# relink_unit points /etc/systemd/system/frank-dragoon.service at the repo
# file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity
+15 -8
View File
@@ -49,14 +49,21 @@ FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_conf
# stays at the image default, which is what was qualified) ------------------
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
HALOGEN_MODEL_ID=frank-dragoon
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
# the page) — a window equal to the ceiling makes every long request a 400.
HALOGEN_CTX=200704
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
# at once. Pool positions buy concurrency; the per-request window never moves
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
# RAM the pool takes is RAM the n-gram table's page cache loses.
# PER-REQUEST ceiling, set to the checkpoint's NATIVE window (262,144 — a
# longer prompt is a 400 naming the limit, never a truncation; stretching
# past native would need HALOGEN_ROPE_YARN and is not on plan). The harness
# `contextWindow` must sit ~45k under THIS number (the 400-zone law on the
# page): with the window at 200,000 the gap is 62,144, which absorbs the
# chars/4 under-count. A window at or near the ceiling makes every long
# request a guaranteed 400.
HALOGEN_CTX=262144
# ONE pool behind both slots. The harness window is 200,000, so 401,408
# still carries TWO maximum harness-length conversations at once
# (2 x 200,000 + one 45,056 budget <= pool + cap headroom); two literal
# ceiling-length requests would queue instead — buying those means raising
# the pool, and RAM the pool takes is RAM the n-gram table's page cache
# loses. Pool positions buy concurrency; the per-request window never moves
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it).
HALOGEN_KV_POOL_POSITIONS=401408
# Two concurrent conversations: the MTP drafter is not slot-locked on this
# engine, so the second slot is capacity, not a speculation tax.
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 401408-position pool at ctx 200704, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 401408-position pool at ctx 262144, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
After=network-online.target
Wants=network-online.target
+18 -14
View File
@@ -52,28 +52,31 @@ Where the contract lives:
The arithmetic (the coupled contract every endpoint's maxTokens comment
encodes). The authoritative `contextWindow`/`maxTokens` pair per model is on the
endpoint page ([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]); what
follows is a **worked example** recomputed from the live files on 2026-10-18 —
a snapshot, not fleet state — with windows 200,000 / 155,648 / 200,704 and
follows is a **worked example** recomputed from the live files on 2026-10-21 —
a snapshot, not fleet state — with windows 200,000 / 200,000 / 200,704 and
maxTokens 16,384 / 45,056 / 16,384 (archon / dragoon / zealot):
1. **Source trigger** = `thresholdPercent × source contextWindow`; at the live
95 % that is 190,000 (archon), 147,866 (dragoon), 190,669 (zealot). The
95 % that is 190,000 (archon and dragoon), 190,669 (zealot). The
bundle clamps the percentage to 1–99 and converts it to a token threshold.
2. **Output budget** = capped at the target model's `maxTokens` — every role
here compacts to frank-zealot, so the budget is its 16,384.
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
window. **At 95 % this no longer holds**: 190,000 + 16,384 = 206,384 >
200,704. The registry's own comments design around an 85 % trigger (archon:
170,000 est × 1.03 + 16,384 ≈ 191,500 ≤ 200,704, `models.yml:49-56`), which
does hold. While `config.yml` and those comments disagree, treat 85 % as the
design point and re-run this arithmetic after ANY window change — a 400 from
the compaction target mid-conversation is the symptom.
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the fitting window. For a
dragoon session the fitting window is the ENGINE ceiling, not a slot: the
190,000 trigger is 62,144 under the 262,144 ceiling, so even ×1.03 est
error plus the 45,056 request budget lands inside — that headroom is the
reason the ceiling moved to native on 2026-10-21. For a session compacting
INTO frank-zealot the window is its 200,704-token slot, and **at 95 % this
does not hold for archon/zealot sources**: 190,000 + 16,384 = 206,384 >
200,704. Treat 85 % as the design point until config.yml and the registry
comments agree, and re-run this arithmetic after ANY window change — a 400
from the compaction target mid-conversation is the symptom.
(Historic 2026-10-05 pairs, at `thresholdPercent: 70`: 140,492 + 36,864 =
177,356 ≤ 200,704 archon; 140,490 + 57,344 ≤ 200,700 dragoon; 140,492 +
32,768 ≤ 200,704 zealot.)
Procedure (recompute from the LIVE files every time — windows rotate with
every swap; the block above is the 2026-10-18 state):
every swap; the block above is the 2026-10-21 state):
1. Read the `config.yml` compaction block + `models.yml` entries for source
and target. Every endpoint's `contextWindow` comment carries its own
@@ -94,9 +97,10 @@ Known reconciliation trap: comments in models.yml have described compaction
as running "on the dankserver endpoint" while `remoteEndpoint` pointed at
dankclaw — the comment/config disagreed. The comments now name the real
target (`compaction.remoteEndpoint` = dankclaw :7771). **The config is
truth; fix comments in the same change.** A fresh instance of the same trap is
open 2026-10-18: every `maxTokens` comment still says "compaction fires at
85 % of the window" while `config.yml:84` sets `thresholdPercent: 95`.
truth; fix comments in the same change.** The 85 %-comments vs 95 %-config
disagreement is not academic: it is why long dragoon sessions kept drawing
400s at the old 200,704 ceiling — the trigger fires late AND the est
under-counts, and a window parked near the ceiling has no room for either.
`maxTokens` never changes for a reslice — it caps output; `contextWindow` caps
input. Boot-shape OOM laddering is a different problem
([[Llama Cpp Serving Sizing]]).
+26 -18
View File
@@ -10,14 +10,15 @@ tags:
- gfx1151
- halogen
- endpoint
timestamp: 2026-10-19
timestamp: 2026-10-21
---
# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
(`2b12716`). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.
(`2b12716`), ceiling raised to the native 262,144 on 2026-10-21 so the harness window is 200,000.
This page replaces every per-era dragoon runbook; the Eras table is the rollback record.
## Current shape
@@ -37,10 +38,12 @@ Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b
unauthenticated token port 8730 stays on container loopback. Engine stdout →
`/home/dank/logs/frank_dragoon.log`, **not** the journal.
- Shape: `HALOGEN_CTX=200704` hard per-request ceiling · `HALOGEN_KV_SLOTS=2` ·
`HALOGEN_KV_POOL_POSITIONS=401408` = slots × ctx, the top of the band `tests/frank_dragoon_unit.bats`
enforces (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob
exists here, the pool is the only KV lever · native window 262,144, `HALOGEN_ROPE_YARN` beyond it.
- Shape: `HALOGEN_CTX=262144` hard per-request ceiling = the checkpoint's native window (raised
2026-10-21 from 200704; `HALOGEN_ROPE_YARN` beyond native is not on plan) · `HALOGEN_KV_SLOTS=2` ·
`HALOGEN_KV_POOL_POSITIONS=401408` inside the band `tests/frank_dragoon_unit.bats` enforces
(`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob
exists here, the pool is the only KV lever · the pool carries TWO maximum harness-length (200k)
conversations at once; two literal ceiling-length requests would queue — that is a pool raise.
- Knobs: `HALOGEN_MAX_TOK=8192` sizes the prefill arena (5.7 GiB) and is only the default for requests
omitting `max_tokens` · `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
pair · MTP depth 2 with the head INSIDE the checkpoint (**`HALOGEN_MTP_HEAD` stays unset**) and
@@ -53,10 +56,11 @@ unauthenticated token port 8730 stays on container loopback. Engine stdout →
(Pin::None)`; no clamp line. Host `MemAvailable` read 22 GiB right after the 2026-10-18 raise —
~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI,
Caddy/observability).
- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648`
(= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`
`reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction
rides [[frank-zealot]].
- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 200000`
— sits 62,144 under the ceiling, clearing the ~45k fit-law gap (400-zone law below);
`tests/frank_dragoon_unit.bats` fails if the page's declared window and `HALOGEN_CTX` drift back
together. `maxTokens: 45056`, `input: [text, image]`, `reasoningContentField: "reasoning_content"`,
`high→xhigh` fold, `extraBody` 1.0/0.95. Compaction rides [[frank-zealot]].
- Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights,
pool, MTP, prompt cache, **no clamp**) + `/health` 200 + a tool call + the 512 px red-PNG probe.
Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it) and `amdgpu.noretry=0` — the env says check it at
@@ -124,14 +128,18 @@ created it — earlier topologies in .
prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in
85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so
re-measure the warm ratio after any pool change.
- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow:
155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`
its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The
3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 —
over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k
target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request
is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 %
under-count. Compaction rides [[frank-zealot]] at ≈ threshold − keepRecent(20k) ≈ 112k input.
- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling, because the harness
clamps `max_tokens` to `contextWindow − est − 4096`, its chars/4 `est` UNDER-counts code/agent
traffic by ~3 %, and the compaction trigger fires at 95 % of the window (`config.yml
thresholdPercent`, while the comments designed around 85 % — check the trigger BEFORE touching a
window; that disagreement is why long sessions kept drawing 400s). The 3 % is measured, not
theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 — over by 354, a 400.
At the live pair (ceiling 262,144 / window 200,000) the worst-case ordinary request is
200,000 − 4,096 + 3 % ≈ 201.8k, and the trigger point is 190,000 est × 1.03 + 45,056 ≈ 240.7k —
both inside the ceiling with ~60k of margin. Compaction rides [[frank-zealot]] at
≈ threshold − keepRecent(20k) ≈ 175k measured against its 200,704-token slot. The seam in
`tests/frank_dragoon_unit.bats` re-runs this arithmetic at the 95 % trigger and reads the window
from THIS page, so ceiling/window pairs that drift back together fail the suite, not a session.
- **Admission reserves prompt + `max_tokens`:** an un-fittable request waits in arrival order, then
503s `engine_busy` for the whole queue after `QUEUE_TIMEOUT` — and the cap/timeout pair moves
together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
+31 -24
View File
@@ -3,7 +3,7 @@
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE
# 200,704-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
# 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0)
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
#
@@ -85,7 +85,7 @@ good_env() {
"FRANK_DRAGOON_VISION_BYTES=0"
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
"HALOGEN_MODEL_ID=frank-dragoon"
"HALOGEN_CTX=200704"
"HALOGEN_CTX=262144"
"HALOGEN_KV_POOL_POSITIONS=401408"
"HALOGEN_KV_SLOTS=2"
"HALOGEN_SPEC_ADAPT=0"
@@ -159,9 +159,11 @@ good_env() {
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
# The pool is ONE budget behind the slots (boot ledger form: "2 slots over
# ONE <pool>-position KV pool, each request up to HALOGEN_CTX"), NOT
# a per-slot private pool. Since 2026-10-18 it is sized pool = slots * ctx
# (401,408 = 2 x 200,704), so two MAXIMUM-length conversations can be
# resident at once — frank-zealot parity on total in-flight context. What
# a per-slot private pool. Since 2026-10-21 the ceiling is the checkpoint's
# native 262,144 while the pool stays 401,408 (raised 2026-10-18): two
# MAXIMUM harness-length (200,000-token) conversations are resident at once;
# two literal ceiling-length requests would queue — widening that is a pool
# raise, not a window move.
# still binds: the pool must back ONE full-length request (below that the
# advertised per-request ceiling would be unreachable, hence pool >= ctx),
# it must not pretend to be more than slots x ctx (pool <= slots * ctx), and
@@ -287,7 +289,7 @@ $(cat "$TEST_TMP/serve.log")"
[[ "$output" == *"--name frank-dragoon-halogen"* ]]
[[ "$output" == *"--publish 7770:8731"* ]]
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
[[ "$output" == *"-e HALOGEN_CTX=200704"* ]]
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]]
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=401408"* ]]
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
@@ -306,27 +308,32 @@ $(cat "$TEST_TMP/serve.log")"
@test "the shipped env keeps the harness fit law inside the ceiling" {
# HALOGEN_CTX is the hard per-request ceiling (a longer prompt is a 400
# naming the limit, never a silent truncation). The law is INVARIANT to
# the value: the harness window must sit ~45k UNDER it, because the
# harness clamps max_tokens = min(45056, window − est − 4096), compacts
# at 85 % of its declared window, and its chars/4 est UNDER-counts the
# rendered prompt by ~3 % on code/agent traffic (measured 4,450 tokens at
# est 152,938 on 2026-10-07 — with window == ceiling, every request over
# ~65 % of the window was a guaranteed 400 before compaction could fire).
# The shipped pair is ctx 200704 / window 155648 (lowered from
# 262144/217088 on 2026-10-17); the gap must absorb the error at BOTH
# monotone maxima of est + budget (est ≤ window − 4096):
# full budget at the compaction threshold: 1.03*132300 + 45056
# clamped budget at the clamp's zero point: 1.03*151552 + 0
# and the compaction request that rides on frank-zealot (its
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
grep -qE '^HALOGEN_CTX=200704$' "$ENV_FILE"
ctx=200704
hw=155648; mt=45056
thr=$(( hw * 85 / 100 ))
# naming the limit, never a silent truncation). The law: the harness
# window must sit ~45k UNDER the ceiling, because the harness clamps
# max_tokens = min(45056, window − est − 4096), compacts at the trigger
# percent of its declared window, and its chars/4 est UNDER-counts the
# rendered prompt by ~3 % on code/agent traffic (measured 4,450 tokens
# at est 152,938 on 2026-10-07 — with window ≈ ceiling, every long
# request is a guaranteed 400 before compaction can fire). The 2026-10-21
# pair: ceiling 262144 (native) under a harness window of 200,000.
# The declared window is read from the endpoint page — the one owner —
# so env, models.yml and page that disagree FAIL HERE, not as a 400
# mid-session (the recurring failure this seam exists for).
grep -qE '^HALOGEN_CTX=262144$' "$ENV_FILE"
ctx="$(sed -nE 's/^HALOGEN_CTX=([0-9]+)$/\1/p' "$ENV_FILE")"
page="$ROOT/serverAppConfigs-docs/LLM/frank-dragoon.md"
grep -qE 'contextWindow: 200000' "$page"
hw="$(grep -oE 'contextWindow: [0-9]+' "$page" | head -1 | grep -oE '[0-9]+')"
mt=45056
# Invariant checked at the live 95 % trigger (config.yml
# thresholdPercent) — strictly stronger than the 85 % design point.
# Both monotone maxima of est + budget (est ≤ window − 4096):
thr=$(( hw * 95 / 100 ))
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
zero=$(( hw - 4096 ))
[ $(( zero * 103 / 100 )) -le "$ctx" ]
# The compaction request rides on frank-zealot's 200,704-token slot:
# input ≈ threshold − keepRecent(20k).
[ $(( (thr - 20000) * 103 / 100 )) -le 200704 ]
}