Frank
a34322aeb0
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
25 lines
2.1 KiB
Markdown
25 lines
2.1 KiB
Markdown
---
|
||
name: omp-harness-window-fit-audit
|
||
description: "Audit an omp harness model entry against an engine's prompt+max_tokens fit check; explains and fixes 400 \"max_tokens does not fit\" at 60-85 % of the declared window."
|
||
---
|
||
|
||
Use when a local endpoint returns 400 `max_tokens N does not fit: prompt is P tokens and the context is C, leaving room for R` on omp, typically at 65-85 % of the declared window.
|
||
|
||
## The three numbers that must agree
|
||
1. Engine hard ceiling: prompt_tokens + max_tokens ≤ window (halogen `HALOGEN_CTX`/slot, llama.cpp per-slot `ctx_size/parallel`, radiance `--max-model-len`). A 400, never truncation.
|
||
2. Harness clamp: `max_tokens = min(maxTokens, contextWindow − est − 4096)`, est = chars/4 over the conversation+tools. The dump in `~/.omp/logs/http-400-requests/*.json` proves it: est = contextWindow − 4096 − max_completion_tokens.
|
||
3. Compaction fires only at `compaction.thresholdPercent` × contextWindow (config.yml; ignore stale reserveTokens/70 % comments).
|
||
|
||
## The trap
|
||
chars/4 under-counts the rendered chat template by ~3 % on code/agent traffic. If contextWindow == engine ceiling, requests with est > 4096/0.03 ≈ 137k are guaranteed 400s below the compaction threshold (65-85 % of a 200k window). maxTokens ≤ 16,384 hides this; 45,056 exposes it.
|
||
|
||
## Fix procedure
|
||
1. Compute required slack: maxTokens + 0.10 × contextWindow (robust to est error).
|
||
2. Raise the engine ceiling to ≥ contextWindow + slack (native window is the natural choice: Qwen3.8-Flash-Next 262,144). Halogen: pool must stay slots × CTX (`HALOGEN_KV_POOL_POSITIONS`, ~28 KiB/position); check memory ledger.
|
||
3. Keep harness contextWindow below the ceiling by ≥ slack (217,088 = 212×1024 for a 262,144 ceiling with maxTokens 45,056).
|
||
4. Verify compaction itself fits its target (remoteEndpoint slot ≥ thresholdPercent × contextWindow − keepRecentTokens(20k) + overhead).
|
||
5. Update env + seam test + models.yml + AGENTS.md together; a single-file change re-creates the drift.
|
||
|
||
## Verification
|
||
Probe a prompt at ≥ 180k tokens with max_tokens = the clamped value; expect completion, not 400.
|