diff --git a/.gitignore b/.gitignore index 8656541..1ee3996 100644 --- a/.gitignore +++ b/.gitignore @@ -37,3 +37,4 @@ __pycache__/ # Editor/OS noise *.swp .DS_Store +third_party/ diff --git a/serverAppConfigs-docs/LLM/halogen-flash-server.md b/serverAppConfigs-docs/LLM/halogen-flash-server.md index 73d3192..8d7f374 100644 --- a/serverAppConfigs-docs/LLM/halogen-flash-server.md +++ b/serverAppConfigs-docs/LLM/halogen-flash-server.md @@ -340,6 +340,16 @@ deployment: request is a 400 rather than a silent truncation. `max_tokens`, `max_completion_tokens` and `max_output_tokens` all work and must agree if more than one is sent. +- **A budget above (window − compaction-threshold) is a guaranteed 400 zone.** + The fit check (`prompt + max_tokens <= HALOGEN_CTX`) refuses, and the harness + only compacts at its own threshold — with OMP at 70 % of a 200,192 window + (140,134), any prompt in (136,192, 140,134] with a 64,000 budget overflows + the window while sitting below the compaction trigger. That collided live on + 2026-09-17 (prompt 138,898 + 64,000 = 202,898 > 200,192). The fix is + harness-side: keep the declared `maxTokens` at or under + `window − threshold` (57,344 chosen) so no pre-compaction request can miss + the fit check. Do not fix it by raising `HALOGEN_CTX` past pool/2 — that + breaks the two-resident-conversations property. - **`reasoning_effort`** is accepted at the top level and inside `chat_template_kwargs`: `minimal`, `low`, `medium`, `high`, `xhigh`. The model's default is `xhigh`. `minimal` aliases the template's `low` and still