181 lines
9.3 KiB
Markdown
181 lines
9.3 KiB
Markdown
---
|
|
name: omp-models-yml-runner-api-placement
|
|
description: "Add or edit a row in ~/.omp/agent/models.yml — non-chat (tts/image/embedding) runners and authenticated/remote providers with command-backed apiKey values — wire the speech role, verify with a poisoned scratch HOME, and diagnose a failing speech device without blaming a healthy endpoint"
|
|
---
|
|
|
|
# Scope
|
|
|
|
`~/.omp/agent/models.yml` (or `models.yaml`) on a workstation: adding a **runner**
|
|
model row (TTS / image / embedding / rerank / video) alongside chat providers,
|
|
adding an **authenticated or off-network** chat provider without committing its
|
|
secret, flipping `config.yml` `modelRoles` at it, and diagnosing a speech device
|
|
that refuses to speak.
|
|
|
|
# The landmine
|
|
|
|
Provider-level `api` accepts **only chat/judge transports**:
|
|
`openai-completions`, `openai-responses`, `openai-codex-responses`,
|
|
`azure-openai-responses`, `anthropic-messages`, `bedrock-converse-stream`,
|
|
`google-generative-ai`, `google-gemini-cli`, `google-vertex`,
|
|
`openrouter-decisions`, `typesafe`.
|
|
|
|
Naming a runner API (`openai-speech`, `openai-images`, `openai-embeddings`, …)
|
|
at **provider** level fails whole-file validation, and the failure mode is not
|
|
scoped to your provider:
|
|
|
|
```
|
|
Warning: models.yml validation failed — custom providers disabled
|
|
Failed to load config file models, Schema error: providers.<id>.api: must be "openai-completions", …
|
|
```
|
|
|
|
Every custom provider disappears from the registry (fleet chat endpoints stop
|
|
resolving) and `omp models` lists only built-in locals. Measured 2026-10-18.
|
|
|
|
# Correct shape
|
|
|
|
Runner api goes on the **model entry only**; `auth: none` satisfies the
|
|
credential requirement; omit `contextWindow`/`maxTokens` (the positivity check
|
|
fires only when they are provided — `omp models` still prints filled defaults,
|
|
which the runner transport never reads).
|
|
|
|
```yaml
|
|
"frank-tts (dankclaw)":
|
|
baseUrl: http://192.168.0.143:7772/v1 # no `api:` here
|
|
auth: none
|
|
models:
|
|
- id: frank-tts
|
|
name: "frank-tts (dankclaw) — cloned voices"
|
|
api: openai-speech # runner api on the model
|
|
input: [text]
|
|
```
|
|
|
|
Roles live in `config.yml` (`modelRoles`), not in `models.yml`. Kind roles
|
|
(`image`, `web`, `speech`, `dictation`, `judge`) take no `:thinking` suffix.
|
|
|
|
# Authenticated / remote providers
|
|
|
|
Provider schema (`src/config/models-config-schema-bundle.ts`): `baseUrl`,
|
|
`apiKey`, `api`, `headers`, `compat`, `authHeader`, `auth`
|
|
(`"apiKey" | "none" | "oauth"`), `discovery`, `models`, `modelOverrides`,
|
|
`transport`. Defining `models` without `apiKey` and without `auth: none|oauth`
|
|
is a validation error, so an authenticated endpoint needs a key *shape* even if
|
|
the value comes from outside the file.
|
|
|
|
**Never put the literal secret in `models.yml`.** `apiKey` (and any `headers`
|
|
value) is resolved by `src/config/resolve-config-value.ts`:
|
|
|
|
- leading `!` → command-backed: `/bin/sh -c "<rest>"`, cwd = project dir,
|
|
10 s timeout, stdout **trimmed**, result cached for the process lifetime
|
|
(failure backs off 30 s);
|
|
- otherwise the string is looked up as an **exact env-var name**; the literal is
|
|
the fallback only when no such env key exists.
|
|
|
|
So the secret lives in a 600 file, and the tracked file stays clean:
|
|
|
|
```yaml
|
|
apiKey: "!cat ~/.config/secrets/openwebui_api_key" # QUOTE it: an unquoted
|
|
# leading ! is a YAML tag
|
|
```
|
|
|
|
Laws: a literal `sk-…` in this repo trips the pre-commit shape gate *and* lands
|
|
the key in every agent context on every box that pulls; rotation means rewrite
|
|
the secret file **then restart the harness** (the `!cmd` value is process-cached,
|
|
so a live session keeps serving the old key and a stale session reads exactly
|
|
like a revoked one); every workstation needs its own copy of the secret file.
|
|
|
|
**A row behind a proxy copies its direct-connected twin's `compat` block**, it
|
|
does not simplify it. Measured through an OpenAI-compatible gateway on
|
|
2026-10-21: SSE forwards `reasoning_content` and `tool_calls` deltas plus final
|
|
`usage` under `stream_options.include_usage`; `image_url` content parts reach
|
|
the vision tower; `chat_template_kwargs` is forwarded (same prompt:
|
|
`enable_thinking` false → 0 reasoning tokens, true → a real block). Dropping
|
|
kwargs/effort maps because "it is only a proxy" silently changes thinking
|
|
behaviour on that path. Check per gateway, never assume either direction.
|
|
|
|
# Verify after every edit (never assume the file loaded)
|
|
|
|
```bash
|
|
omp models --kind tts --no-extensions # the new row must appear
|
|
omp models --no-extensions | grep -E '<each fleet chat model>' # and still be listed
|
|
omp models | grep -c 'validation failed' # must be 0
|
|
```
|
|
|
|
A run that lists only built-ins means your row broke the file — revert before
|
|
yielding.
|
|
|
|
**Poisoned-scratch-HOME proof** for a remote/credential row (the only test that
|
|
shows a success came through the new path and not from a LAN endpoint):
|
|
|
|
```bash
|
|
mkdir -p /tmp/ompx/.omp/agent /tmp/ompx/.config
|
|
cp agent/models.yml agent/config.yml /tmp/ompx/.omp/agent/
|
|
ln -s ~/.config/secrets /tmp/ompx/.config/secrets # REQUIRED: ~ in a !cmd value
|
|
# expands to the scratch $HOME
|
|
sed -i 's#http://192\.168\.0\.[0-9]*:[0-9]*/v1#http://127.0.0.1:9/v1#g' /tmp/ompx/.omp/agent/models.yml
|
|
HOME=/tmp/ompx omp --model '<provider>/<id>:off' --tools bash --no-session --no-skills \
|
|
--no-rules --no-extensions --no-pty --no-lsp --no-title --cwd /tmp -p '<ask for a real tool call>'
|
|
```
|
|
|
|
Then `rm -rf /tmp/ompx` — that tree holds endpoint URLs. Cross-check the server
|
|
side before believing the client's echo (gateway access log / container log line
|
|
for the request you just made), and confirm the change never hit the live
|
|
checkout until you mean it to: `git -C <live checkout> status --porcelain agent/models.yml`
|
|
plus a grep for the new provider name.
|
|
|
|
# Speech transport contract (`openai-speech`)
|
|
|
|
Body is `{model, input, response_format, voice?, speed?}`; `voice` is forwarded
|
|
**only** when the caller passed `voice_id`; `sample_rate`/`bit_rate` **throw**
|
|
(`openai-speech does not support sampleRate`). Codec comes from the
|
|
`output_path` suffix; 15 000-character cap. An unknown `voice` name is a server
|
|
error, not a fallback — and `auth: none` makes a dead endpoint read as
|
|
*available*, so point `modelRoles.speech` at such a provider only after it has
|
|
answered a real synthesis.
|
|
|
|
A gateway's own `/api/v1/audio/speech` route is NOT an equivalent: OpenWebUI
|
|
substitutes its own voice enum before forwarding, so a cloned pack name comes
|
|
back 400 from the upstream. Keep `modelRoles.speech` on the direct endpoint.
|
|
|
|
# Three speech-role laws (bundle-read + measured 2026-10-18)
|
|
|
|
1. **No setting carries a voice for `openai-speech`.** The transport builds the
|
|
body with `voice` from the call's `voice_id` and nothing else.
|
|
`tts.localVoice` is the *local Kokoro backend's* key; `speech.voice` is the TUI's
|
|
**speak-aloud** voice, an enum of Kokoro names that never reaches an HTTP
|
|
request (setting it to a pack name breaks the vocalizer and does nothing
|
|
for synthesis — read `cfgSpeechVoice` in `dist/cli.js` before believing
|
|
otherwise). So a voice-less call is the harness's **normal** shape: a
|
|
voice-required endpoint needs a server-side default (fleet answer:
|
|
`DEFAULT_VOICE` in `scripts/frank-tts.env`, journal marks the line
|
|
`(default)`, unknown names still 400 with the list).
|
|
2. **An explicit role has no fallback candidate.** `modelRoles.speech` set to one
|
|
model yields exactly one candidate, so a dead endpoint or an unresolvable
|
|
selector fails with *"No available speech model matches the speech role"*
|
|
instead of degrading to Kokoro. Loud-by-design; do not "fix" it by adding a
|
|
fallback in the client.
|
|
3. **`models.yml` loads at process start; `config.yml` is read live.** A session
|
|
older than the new row fails exactly like a dead endpoint while a fresh
|
|
process succeeds in the same minute.
|
|
|
|
# Diagnosing "speech device won't speak" — in this order
|
|
|
|
1. `curl` the endpoint directly with a real voice name. 200 + audio ⇒ the service
|
|
is fine and the fault is harness-side; stop touching the server.
|
|
2. Control test in the *same* session: point `modelRoles.speech` at
|
|
`local/kokoro` and call the device. It works ⇒ `config.yml` is live-read and
|
|
the runner selector or its voice argument is the problem, not the endpoint.
|
|
Revert the role immediately after.
|
|
3. Distinguish stale registry from dead endpoint with a fresh process:
|
|
`omp -p --no-session --no-extensions --no-skills --no-rules --tools tts --cwd /tmp
|
|
'Call the tts tool exactly once with text="…" output_path=/tmp/x.wav
|
|
voice_id=<name> — then reply with the tool result line.'`
|
|
Succeeds ⇒ the long-lived session predates the `models.yml` row; restart it.
|
|
4. Cross-check server-side, never trust the client's echo: the façade/access log
|
|
line and the engine's own ledger must carry the same byte count the tool
|
|
printed. (`model=…` in a tool result is what the client *requested*; a server
|
|
line is the fact.)
|
|
5. For harness-behaviour claims, read the bundle rather than reasoning from the
|
|
tool description: `/home/blank/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`,
|
|
e.g. `grep -o -E '.{300}response_format:[a-zA-Z$]+.{250}'` for the speech body
|
|
builder, `grep -o -E 'cfgSpeechVoice:.{0,400}'` for what a settings key feeds.
|