Frank
872e86399a
config.yml: vision -> llamacpp (dankclaw)/frank-large:auto, tiny -> llamacpp (dankserver)/frank-small:off, task -> llamacpp (dankclaw)/frank-large:medium (was flm dankclaw-npu), matching the flm (dankclaw-npu) provider removal in models.yml. Memories/managed skills: rollouts and learned notes from the frank-large KV-pool + sampler work and the noctalia recolor; merged the duplicate noctalia-palette-recolor skill into noctalia-custom-palette-recolor (adds the dotfiles-tracking step and the +show-config delta gotcha) and dropped the superseded legacy memory skills whose ghostty USR1 reload recipe is wrong for this fleet.
61 lines
3.1 KiB
Markdown
61 lines
3.1 KiB
Markdown
---
|
|
name: omp-per-model-sampler-config
|
|
description: "Set per-model sampler params and preserve_thinking in OMP's models.yml via compat.extraBody, and verify the exact wire body with an echo-capture profile."
|
|
---
|
|
|
|
# OMP Per-Model Sampler Config (`compat.extraBody`)
|
|
|
|
`~/.omp/agent/models.yml` has **no per-model sampler keys**. Sampling is global (`config.yml`: `temperature`, `topP`, `topK`, `minP`, `presencePenalty`, `repetitionPenalty`; default `-1` = **not sent**). The per-model hook is `compat.extraBody`.
|
|
|
|
## Why it works
|
|
`applyOpenAIExtraBody` runs **last** in the openai-completions builder — after `applyChatCompletionsCompatPolicy` — via `Object.assign(params, extraBody)`. So extraBody overrides global settings and compat-policy output.
|
|
|
|
## Recipe
|
|
Use **wire names**, not OMP camelCase:
|
|
|
|
```yaml
|
|
- id: frank-large
|
|
compat:
|
|
reasoningEffortMap:
|
|
max: xhigh
|
|
extraBody:
|
|
temperature: 1.0
|
|
top_p: 0.95
|
|
top_k: 20
|
|
min_p: 0.0
|
|
presence_penalty: 0.0
|
|
repetition_penalty: 1.0
|
|
preserve_thinking: true
|
|
```
|
|
|
|
## Two traps
|
|
|
|
1. **`preserve_thinking` is not user-settable.** `qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the *auto-detected* `thinkingFormat` + local baseUrl, **before** `applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in models.yml does **not** enable it. Custom-named models (`frank-large`) match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
|
2. **Never put `chat_template_kwargs` in `extraBody`.** `Object.assign` is shallow — it replaces the whole object and silently drops the `reasoning_effort` kwarg the compat policy set. Top-level `preserve_thinking` is honoured by halogen; use that.
|
|
|
|
## Gotcha: silent greedy decode
|
|
halogen-flash-server has **no server-side sampler defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the endpoint decodes greedy while looking configured.
|
|
|
|
## Verify the exact wire body (echo-capture)
|
|
Don't trust the YAML — capture what OMP actually emits.
|
|
|
|
1. Echo server on `127.0.0.1:18099` that dumps the JSON body to a file and returns a canned `chat.completion` (include `reasoning_content` so thinking paths exercise).
|
|
2. Copy the **real** models.yml into a throwaway profile with only the baseUrl swapped:
|
|
```bash
|
|
sed 's|http://<host>:7770/v1|http://127.0.0.1:18099/v1|' \
|
|
~/.omp/agent/models.yml > ~/.omp/profiles/wiretest/agent/models.yml
|
|
```
|
|
3. Run and inspect:
|
|
```bash
|
|
omp --profile wiretest --model "<provider>/<model>" -p "hi" \
|
|
--no-tools --no-session --no-extensions --no-skills --no-rules
|
|
jq 'del(.messages, .tools)' last-body.json
|
|
```
|
|
Confirm every intended field is present with the right value.
|
|
|
|
## Verify a field is *honoured*, not just accepted
|
|
Unknown fields often return 200 anyway. Prove behavior with a **prompt-token delta**: replay an assistant turn carrying a long `reasoning_content`, then compare `usage.prompt_tokens` with the flag true vs false. A large delta means the field changed template rendering; zero delta means it was ignored.
|
|
|
|
## Cleanup
|
|
Trash the probe profile and echo server (`profiles/.AGENTTRASH`), never `rm`.
|