frank-dragoon: the harness rewrites reasoning_effort values the engine accepts, and caps max at xhigh #62

Closed
opened 2026-10-21 01:24:07 -07:00 by dank · 2 comments
Owner

What is wrong

agent/models.yml tells the harness what the frank-dragoon engine accepts. Its reasoningEffortMap encodes a rejection rule that the engine does not have, and the encoding costs thinking levels.

The entry (provider llamacpp (dankclaw), :7770, halogen flash-server 0.16.2):

reasoningEffortMap:
  minimal: low
  low: low
  medium: medium
  high: xhigh
  max: xhigh

with the comment "any reasoning_effort outside {xhigh, medium, low} is rejected, not clamped — the fold stays mandatory".

Measured against the live endpoint on 2026-10-21 — the engine's own 400 names its accepted set:

reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none

So minimal, high, xhigh, max and none are all accepted; the set is not {xhigh, medium, low}. Consequences:

  • minimal → low and high → xhigh rewrite requests the engine would have taken natively: asking for minimal gets low, asking for high gets xhigh. The thinking budget is not what was asked for.
  • max → xhigh caps the largest level. The engine treats max as distinct — measured thinking length on one prompt at max_tokens 2048: low 766 reasoning tokens, xhigh 614, max 1049, the block closed at 1024 (reasoning_closed_at: 1024). The max thinking ceiling is unreachable from the harness.
  • none is the one level that changes kind: reasoning_effort: none returns
    reasoning_tokens: 0 — thinking off entirely (measured twice, content only). It does NOT need a map
    entry, because omp never puts off on the wire: the bundle turns reasoning off through a dedicated
    disable mode (T.effort==="off" ? !1 : …, then reasoningDisableMode, whose default comes from
    thinkingFormat — qwen family → qwen-enable-thinking-false). Measured on this engine, that path
    works: chat_template_kwargs {"enable_thinking": false} and top-level enable_thinking: false both
    return reasoning_tokens: 0. So thinking-off needs no knob here, and a none/off map entry would
    be dead config — the contract belongs in the comment, not in the map.
  • The bundle applies the map as compat.reasoningEffortMap?.[effort] ?? … ?? effort (verified in the pinned dist/cli.js: qIi(e,t){return e.compat.reasoningEffortMap?.[t]??e.thinking?.effortMap?.[t]??t}) — an unmapped level passes through verbatim. The fold protects nothing from rejection; it only renames levels the engine already spells.

Two knobs were verified correct and stay untouched: input: [text, image] (512 px solid-red PNG → answer "Red", 282 prompt tokens, tower priced at 0.8 GiB in the boot ledger) and the sampler pair 1.0 / 0.95. top_k is the one asymmetry: the comment claims the wire matches the server sampler 1.0 / 0.95 / 20, but top_k is never sent — that value comes from HALOGEN_TOP_K in scripts/frank-dragoon.env, so a change there silently changes sampling under a comment that says the recipe is pinned on the wire.

Also on the same block: the rename note reads "Model id renamed from frank-dragoon on 2026-10-08" — self-referential, because the callsign rename (0cb0021) replaced the old id inside its own explanation. git show eebcfbf:agent/models.yml has the truth: :7770 on dankclaw was frank-large, :7771 was frank-small, :235:7770 was frank-medium. All three entries carry the mangled sentence, so the record an agent needs to resolve an old id is gone from all of them.

Fix

  • reasoningEffortMap becomes the measured identity — minimal/low/medium/high/xhigh/max to themselves. No off/none entry: the suppression path above already works, and a mapping that never fires is the dead config this repo keeps deleting.
  • The comment carries the engine's own allow-list sentence, the measured level difference, the verified thinking-off contract, and the probe that reproduces all three, so the next reader re-measures instead of inheriting a guess.
  • Restore the three pre-rename ids.
  • Pin top_k: 20 in extraBody (accepted by the engine today; makes the comment true and the recipe independent of the env).

Not here

The engine-side contract sentence belongs on serverAppConfigs-docs/LLM/frank-dragoon.md — that repo owns the engine's shape. Cross-linked when it lands.


cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05

## What is wrong `agent/models.yml` tells the harness what the frank-dragoon engine accepts. Its `reasoningEffortMap` encodes a rejection rule that the engine does not have, and the encoding costs thinking levels. The entry (provider `llamacpp (dankclaw)`, `:7770`, halogen flash-server 0.16.2): ```yaml reasoningEffortMap: minimal: low low: low medium: medium high: xhigh max: xhigh ``` with the comment "any `reasoning_effort` outside {xhigh, medium, low} is rejected, not clamped — the fold stays mandatory". Measured against the live endpoint on 2026-10-21 — the engine's own 400 names its accepted set: ``` reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none ``` So `minimal`, `high`, `xhigh`, `max` and `none` are all **accepted**; the set is not {xhigh, medium, low}. Consequences: - `minimal → low` and `high → xhigh` rewrite requests the engine would have taken natively: asking for `minimal` gets `low`, asking for `high` gets `xhigh`. The thinking budget is not what was asked for. - `max → xhigh` caps the largest level. The engine treats `max` as distinct — measured thinking length on one prompt at `max_tokens 2048`: `low` 766 reasoning tokens, `xhigh` 614, `max` **1049, the block closed at 1024** (`reasoning_closed_at: 1024`). The `max` thinking ceiling is unreachable from the harness. - `none` is the one level that changes kind: `reasoning_effort: none` returns `reasoning_tokens: 0` — thinking off entirely (measured twice, content only). It does NOT need a map entry, because omp never puts `off` on the wire: the bundle turns reasoning off through a dedicated disable mode (`T.effort==="off" ? !1 : …`, then `reasoningDisableMode`, whose default comes from `thinkingFormat` — qwen family → `qwen-enable-thinking-false`). Measured on this engine, that path works: `chat_template_kwargs {"enable_thinking": false}` and top-level `enable_thinking: false` both return `reasoning_tokens: 0`. So thinking-off needs no knob here, and a `none`/`off` map entry would be dead config — the contract belongs in the comment, not in the map. - The bundle applies the map as `compat.reasoningEffortMap?.[effort] ?? … ?? effort` (verified in the pinned `dist/cli.js`: `qIi(e,t){return e.compat.reasoningEffortMap?.[t]??e.thinking?.effortMap?.[t]??t}`) — an unmapped level passes through **verbatim**. The fold protects nothing from rejection; it only renames levels the engine already spells. Two knobs were verified correct and stay untouched: `input: [text, image]` (512 px solid-red PNG → answer "Red", 282 prompt tokens, tower priced at 0.8 GiB in the boot ledger) and the sampler pair `1.0 / 0.95`. `top_k` is the one asymmetry: the comment claims the wire matches the server sampler `1.0 / 0.95 / 20`, but `top_k` is never sent — that value comes from `HALOGEN_TOP_K` in `scripts/frank-dragoon.env`, so a change there silently changes sampling under a comment that says the recipe is pinned on the wire. Also on the same block: the rename note reads "Model id renamed from **frank-dragoon** on 2026-10-08" — self-referential, because the callsign rename (`0cb0021`) replaced the old id inside its own explanation. `git show eebcfbf:agent/models.yml` has the truth: `:7770` on dankclaw was **frank-large**, `:7771` was **frank-small**, `:235:7770` was **frank-medium**. All three entries carry the mangled sentence, so the record an agent needs to resolve an old id is gone from all of them. ## Fix - `reasoningEffortMap` becomes the measured identity — `minimal/low/medium/high/xhigh/max` to themselves. No `off`/`none` entry: the suppression path above already works, and a mapping that never fires is the dead config this repo keeps deleting. - The comment carries the engine's own allow-list sentence, the measured level difference, the verified thinking-off contract, and the probe that reproduces all three, so the next reader re-measures instead of inheriting a guess. - Restore the three pre-rename ids. - Pin `top_k: 20` in `extraBody` (accepted by the engine today; makes the comment true and the recipe independent of the env). ## Not here The engine-side contract sentence belongs on `serverAppConfigs-docs/LLM/frank-dragoon.md` — that repo owns the engine's shape. Cross-linked when it lands. --- cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05
Author
Owner

PR open: #63 — fix(#62): frank-dragoon's reasoning_effort contract, measured instead of assumed (fix/62-dragoon-effort-contract → main).

Two corrections to this issue while the work was in flight, both from capturing the wire instead of trusting the config:

  • The plan to add off: none to the map is dropped. off never reaches the wire — omp disables reasoning through reasoningDisableMode, whose default sends the lowest effort, so :off was arriving as reasoning_effort: "minimal" (thinking still running). The fix is reasoningDisableMode: none-effort; measured after the change, :off arrives as "none" with reasoning_tokens: 0 and no thinking block.
  • max stays clamped to xhigh even with an identity map: the harness clamps the requested effort to its own ladder for this model before the map applies. Recorded in the comment as the engine-side half of the contract; no shim invented to fake it.

Gates: full suite green, kb-lint 0, YAML parses, and the four efforts captured on the wire (off/high/medium/max → none/high/medium/xhigh, top_k: 20 on every request).


cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05

PR open: https://gitea.franksdank.cloud/dank/command-center/pulls/63 — **fix(#62): frank-dragoon's reasoning_effort contract, measured instead of assumed** (`fix/62-dragoon-effort-contract` → `main`). Two corrections to this issue while the work was in flight, both from capturing the wire instead of trusting the config: - The plan to add `off: none` to the map is **dropped**. `off` never reaches the wire — omp disables reasoning through `reasoningDisableMode`, whose default sends the lowest effort, so `:off` was arriving as `reasoning_effort: "minimal"` (thinking still running). The fix is `reasoningDisableMode: none-effort`; measured after the change, `:off` arrives as `"none"` with `reasoning_tokens: 0` and no thinking block. - `max` **stays clamped to `xhigh`** even with an identity map: the harness clamps the requested effort to its own ladder for this model before the map applies. Recorded in the comment as the engine-side half of the contract; no shim invented to fake it. Gates: full suite green, `kb-lint` 0, YAML parses, and the four efforts captured on the wire (`off/high/medium/max` → `none/high/medium/xhigh`, `top_k: 20` on every request). --- cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05
Author
Owner

Merged: 6e89c63 (--no-ff, PR #63: #63), and landed in the deployment checkout by 22d2f3b — ~/.omp is at origin/main and the suite is green there.


cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05

Merged: `6e89c63` (`--no-ff`, PR #63: https://gitea.franksdank.cloud/dank/command-center/pulls/63), and landed in the deployment checkout by `22d2f3b` — `~/.omp` is at `origin/main` and the suite is green there. --- cwd: /home/blank/.omp/wt/wt-20261019-125509-7c8a99d · session: 01a155bb-6474-7366-be9d-629f74af0e05
dank closed this issue 2026-10-21 01:45:48 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dank/command-center#62