fix(#62): frank-dragoon's reasoning_effort contract, measured instead of assumed #63

Merged
dank merged 1 commits from fix/62-dragoon-effort-contract into main 2026-10-21 01:44:23 -07:00
Owner

Work list: #62

One file — agent/models.yml, the frank-dragoon entry — and every number in this body came off the endpoint, not off the comment that was already in the file.

What was wrong

The entry folded every reasoning effort before the wire (minimal→low, high→xhigh, max→xhigh) under a comment asserting flash-server rejects anything outside {xhigh, medium, low}. It does not. Its own 400 names the set:

reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none

and the harness passes an unmapped level through verbatim (compat.reasoningEffortMap?.[effort] ?? effort, read in the pinned dist/cli.js). The fold protected nothing and redefined what callers asked for. Levels are not cosmetic on this engine — thinking length on one prompt at max_tokens 2048: low 766 reasoning tokens, xhigh 614, max 1049, block closed at 1024 (reasoning_closed_at).

What changed

Setting Before After Why
reasoningEffortMap 5 folding entries identity (minimal/low/medium/high/xhigh/max) engine spells every level; folding changed the requested budget
reasoningDisableMode unset → lowest-effort none-effort :off was arriving as reasoning_effort: "minimal", and minimal still thinks
extraBody.top_k absent 20 comment claimed the wire matched the server sampler 1.0/0.95/20 while top_k was never sent
rename comments (3 entries) "renamed from frank-dragoon" (self-referential) frank-medium / frank-large / frank-small the 2026-10-08 callsign rename overwrote the old id inside its own explanation

Evidence: the wire, captured

Config claims about what the harness sends are worth exactly their last capture, so this was verified by pointing a scratch HOME at a logging proxy on :7799 forwarding to :7770, running one-shot omp with this branch's models.yml, and reading the request bodies:

requested wire before wire after
:off minimal (thinking still ran — canned 26-token block) none (reasoning_tokens: 0, no thinking block rendered)
:high xhigh high
:medium medium medium
:max xhigh xhigh — see below
top_k absent 20 on every request

max stays clamped, and this branch does not pretend otherwise. The harness clamps the requested effort to its own ladder for this model before the map applies, and that ladder tops out at xhigh. The max row is the engine-side half of the contract, ready when the ladder gains it; nothing in models.yml can force it, and no shim was invented to fake it.

Verified correct, left alone

  • input: [text, image] — 512 px solid-red PNG returns "Red" at 282 prompt tokens; the tower is priced at 0.8 GiB in the deployed boot ledger.
  • contextWindow: 155648 / maxTokens: 45056 against HALOGEN_CTX=200704 — the 400-zone law (window = ceiling − 45k) holds, engine and env agree with the vault page.
  • reasoningContentField: "reasoning_content" — emitted, measured.
  • The suppression switch itself needed no new knob beyond the mode: chat_template_kwargs {"enable_thinking": false} and top-level enable_thinking: false both return reasoning_tokens: 0, and the engine accepts top_k / min_p / presence_penalty / repetition_penalty (all 200).

Deliberately absent

  • No off: none map row. The first draft of this fix added one; the capture killed it — off never reaches the wire, omp disables reasoning through reasoningDisableMode, so the row would have been the exact dead config this repo keeps deleting. The contract belongs in the comment, where it now is.
  • No test. Pinning models.yml to value: x is a configuration assertion, not a test; the re-measurement recipe is in the comment (one curl for the allow-list, max_tokens 2048 for the levels, logging proxy for the wire).

Gates

Full suite green (0 failures across the suite) · kb-lint 0 violations · YAML parses and the parsed map/extraBody/window read back as intended · harness loaded the file and produced the table above.

Not here

The engine-side sentence (reasoning_effort allow-list, the max→1024 thinking ceiling, thinking-off spellings) belongs on serverAppConfigs-docs/LLM/frank-dragoon.md — that repo owns the engine's shape. Say the word and I'll land it there too.


cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05

Work list: https://gitea.franksdank.cloud/dank/command-center/issues/62 One file — `agent/models.yml`, the frank-dragoon entry — and every number in this body came off the endpoint, not off the comment that was already in the file. ## What was wrong The entry folded every reasoning effort before the wire (`minimal→low`, `high→xhigh`, `max→xhigh`) under a comment asserting flash-server rejects anything outside `{xhigh, medium, low}`. It does not. Its own 400 names the set: ``` reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none ``` and the harness passes an unmapped level through verbatim (`compat.reasoningEffortMap?.[effort] ?? effort`, read in the pinned `dist/cli.js`). The fold protected nothing and redefined what callers asked for. Levels are not cosmetic on this engine — thinking length on one prompt at `max_tokens 2048`: `low` 766 reasoning tokens, `xhigh` 614, `max` **1049, block closed at 1024** (`reasoning_closed_at`). ## What changed | Setting | Before | After | Why | |---|---|---|---| | `reasoningEffortMap` | 5 folding entries | identity (`minimal/low/medium/high/xhigh/max`) | engine spells every level; folding changed the requested budget | | `reasoningDisableMode` | unset → lowest-effort | `none-effort` | `:off` was arriving as `reasoning_effort: "minimal"`, and `minimal` still thinks | | `extraBody.top_k` | absent | `20` | comment claimed the wire matched the server sampler `1.0/0.95/20` while `top_k` was never sent | | rename comments (3 entries) | "renamed from frank-dragoon" (self-referential) | `frank-medium` / `frank-large` / `frank-small` | the 2026-10-08 callsign rename overwrote the old id inside its own explanation | ## Evidence: the wire, captured Config claims about what the harness *sends* are worth exactly their last capture, so this was verified by pointing a scratch `HOME` at a logging proxy on `:7799` forwarding to `:7770`, running one-shot `omp` with this branch's `models.yml`, and reading the request bodies: | requested | wire before | wire after | |---|---|---| | `:off` | `minimal` (thinking still ran — canned 26-token block) | **`none`** (`reasoning_tokens: 0`, no thinking block rendered) | | `:high` | `xhigh` | `high` | | `:medium` | `medium` | `medium` | | `:max` | `xhigh` | `xhigh` — see below | | `top_k` | absent | `20` on every request | **`max` stays clamped, and this branch does not pretend otherwise.** The harness clamps the requested effort to its own ladder for this model *before* the map applies, and that ladder tops out at `xhigh`. The `max` row is the engine-side half of the contract, ready when the ladder gains it; nothing in `models.yml` can force it, and no shim was invented to fake it. ## Verified correct, left alone - `input: [text, image]` — 512 px solid-red PNG returns **"Red"** at 282 prompt tokens; the tower is priced at 0.8 GiB in the deployed boot ledger. - `contextWindow: 155648` / `maxTokens: 45056` against `HALOGEN_CTX=200704` — the 400-zone law (`window = ceiling − 45k`) holds, engine and env agree with the vault page. - `reasoningContentField: "reasoning_content"` — emitted, measured. - The suppression switch itself needed no new knob beyond the mode: `chat_template_kwargs {"enable_thinking": false}` and top-level `enable_thinking: false` both return `reasoning_tokens: 0`, and the engine accepts `top_k / min_p / presence_penalty / repetition_penalty` (all 200). ## Deliberately absent - **No `off: none` map row.** The first draft of this fix added one; the capture killed it — `off` never reaches the wire, omp disables reasoning through `reasoningDisableMode`, so the row would have been the exact dead config this repo keeps deleting. The contract belongs in the comment, where it now is. - **No test.** Pinning `models.yml` to `value: x` is a configuration assertion, not a test; the re-measurement recipe is in the comment (one curl for the allow-list, max_tokens 2048 for the levels, logging proxy for the wire). ## Gates Full suite green (0 failures across the suite) · `kb-lint` 0 violations · YAML parses and the parsed map/`extraBody`/window read back as intended · harness loaded the file and produced the table above. ## Not here The engine-side sentence (`reasoning_effort` allow-list, the `max`→1024 thinking ceiling, thinking-off spellings) belongs on `serverAppConfigs-docs/LLM/frank-dragoon.md` — that repo owns the engine's shape. Say the word and I'll land it there too. --- cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05
dank added 1 commit 2026-10-21 01:28:52 -07:00
The entry rewrote every effort before the wire: `minimal -> low`, `high -> xhigh`,
`max -> xhigh`, under a comment claiming flash-server rejects anything outside
{xhigh, medium, low}. The engine says otherwise, in its own 400:

    reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none

and the harness passes an unmapped level through verbatim
(`compat.reasoningEffortMap?.[effort] ?? effort`, read in the pinned `dist/cli.js`). So the
fold protected nothing and silently changed what the caller asked for. The levels are not
cosmetic either — thinking length on one prompt at `max_tokens 2048`: `low` 766 reasoning
tokens, `xhigh` 614, `max` 1049 with the block closed at 1024.

The map is identity. Two things came out of capturing the wire (scratch HOME + a logging
proxy forwarding :7799 to :7770) that reading the config could not have told me:

- `:off` does not switch thinking off. The default disable mode sends the lowest effort,
  so `off` left as `reasoning_effort: "minimal"` — and `minimal` still thinks, a canned
  26-token block on every request. `reasoningDisableMode: none-effort` makes the harness
  send `"none"`, which the engine honours as a real switch: `reasoning_tokens: 0`, answer
  only, no thinking block rendered. Verified after the change, not before it.
- `:max` still arrives as `xhigh`: the harness clamps the requested effort to its own
  ladder for this model before the map applies. Recorded in the comment as the engine-side
  half of the contract rather than papered over — nothing in models.yml forces it.

Also pinned `top_k: 20` (accepted; the comment claimed the wire matched the server sampler
while `top_k` was never sent, leaving the recipe one env edit away from drifting), and
restored the pre-callsign model ids `frank-medium` / `frank-large` / `frank-small` — the
2026-10-08 rename had replaced the old id inside its own explanation, leaving "renamed
from frank-dragoon" three times and destroying the record needed to resolve an old id.

Verified: full suite green (0 failures), `kb-lint` 0 violations, YAML parses, and the four
efforts `off/high/medium/max` captured on the wire as `none/high/medium/xhigh` with
`top_k: 20` on every request. Refs #62
dank merged commit 6e89c63e5d into main 2026-10-21 01:44:23 -07:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dank/command-center#63