Files

9.3 KiB

name, description
name description
omp-models-yml-runner-api-placement Add or edit a row in ~/.omp/agent/models.yml — non-chat (tts/image/embedding) runners and authenticated/remote providers with command-backed apiKey values — wire the speech role, verify with a poisoned scratch HOME, and diagnose a failing speech device without blaming a healthy endpoint

Scope

~/.omp/agent/models.yml (or models.yaml) on a workstation: adding a runner model row (TTS / image / embedding / rerank / video) alongside chat providers, adding an authenticated or off-network chat provider without committing its secret, flipping config.yml modelRoles at it, and diagnosing a speech device that refuses to speak.

The landmine

Provider-level api accepts only chat/judge transports: openai-completions, openai-responses, openai-codex-responses, azure-openai-responses, anthropic-messages, bedrock-converse-stream, google-generative-ai, google-gemini-cli, google-vertex, openrouter-decisions, typesafe.

Naming a runner API (openai-speech, openai-images, openai-embeddings, …) at provider level fails whole-file validation, and the failure mode is not scoped to your provider:

Warning: models.yml validation failed — custom providers disabled
  Failed to load config file models, Schema error: providers.<id>.api: must be "openai-completions", …

Every custom provider disappears from the registry (fleet chat endpoints stop resolving) and omp models lists only built-in locals. Measured 2026-10-18.

Correct shape

Runner api goes on the model entry only; auth: none satisfies the credential requirement; omit contextWindow/maxTokens (the positivity check fires only when they are provided — omp models still prints filled defaults, which the runner transport never reads).

  "frank-tts (dankclaw)":
    baseUrl: http://192.168.0.143:7772/v1   # no `api:` here
    auth: none
    models:
      - id: frank-tts
        name: "frank-tts (dankclaw) — cloned voices"
        api: openai-speech                   # runner api on the model
        input: [text]

Roles live in config.yml (modelRoles), not in models.yml. Kind roles (image, web, speech, dictation, judge) take no :thinking suffix.

Authenticated / remote providers

Provider schema (src/config/models-config-schema-bundle.ts): baseUrl, apiKey, api, headers, compat, authHeader, auth ("apiKey" | "none" | "oauth"), discovery, models, modelOverrides, transport. Defining models without apiKey and without auth: none|oauth is a validation error, so an authenticated endpoint needs a key shape even if the value comes from outside the file.

Never put the literal secret in models.yml. apiKey (and any headers value) is resolved by src/config/resolve-config-value.ts:

  • leading ! → command-backed: /bin/sh -c "<rest>", cwd = project dir, 10 s timeout, stdout trimmed, result cached for the process lifetime (failure backs off 30 s);
  • otherwise the string is looked up as an exact env-var name; the literal is the fallback only when no such env key exists.

So the secret lives in a 600 file, and the tracked file stays clean:

    apiKey: "!cat ~/.config/secrets/openwebui_api_key"   # QUOTE it: an unquoted
                                                         # leading ! is a YAML tag

Laws: a literal sk-… in this repo trips the pre-commit shape gate and lands the key in every agent context on every box that pulls; rotation means rewrite the secret file then restart the harness (the !cmd value is process-cached, so a live session keeps serving the old key and a stale session reads exactly like a revoked one); every workstation needs its own copy of the secret file.

A row behind a proxy copies its direct-connected twin's compat block, it does not simplify it. Measured through an OpenAI-compatible gateway on 2026-10-21: SSE forwards reasoning_content and tool_calls deltas plus final usage under stream_options.include_usage; image_url content parts reach the vision tower; chat_template_kwargs is forwarded (same prompt: enable_thinking false → 0 reasoning tokens, true → a real block). Dropping kwargs/effort maps because "it is only a proxy" silently changes thinking behaviour on that path. Check per gateway, never assume either direction.

Verify after every edit (never assume the file loaded)

omp models --kind tts --no-extensions      # the new row must appear
omp models --no-extensions | grep -E '<each fleet chat model>'   # and still be listed
omp models | grep -c 'validation failed'   # must be 0

A run that lists only built-ins means your row broke the file — revert before yielding.

Poisoned-scratch-HOME proof for a remote/credential row (the only test that shows a success came through the new path and not from a LAN endpoint):

mkdir -p /tmp/ompx/.omp/agent /tmp/ompx/.config
cp agent/models.yml agent/config.yml /tmp/ompx/.omp/agent/
ln -s ~/.config/secrets /tmp/ompx/.config/secrets   # REQUIRED: ~ in a !cmd value
                                                    # expands to the scratch $HOME
sed -i 's#http://192\.168\.0\.[0-9]*:[0-9]*/v1#http://127.0.0.1:9/v1#g' /tmp/ompx/.omp/agent/models.yml
HOME=/tmp/ompx omp --model '<provider>/<id>:off' --tools bash --no-session --no-skills \
  --no-rules --no-extensions --no-pty --no-lsp --no-title --cwd /tmp -p '<ask for a real tool call>'

Then rm -rf /tmp/ompx — that tree holds endpoint URLs. Cross-check the server side before believing the client's echo (gateway access log / container log line for the request you just made), and confirm the change never hit the live checkout until you mean it to: git -C <live checkout> status --porcelain agent/models.yml plus a grep for the new provider name.

Speech transport contract (openai-speech)

Body is {model, input, response_format, voice?, speed?}; voice is forwarded only when the caller passed voice_id; sample_rate/bit_rate throw (openai-speech does not support sampleRate). Codec comes from the output_path suffix; 15 000-character cap. An unknown voice name is a server error, not a fallback — and auth: none makes a dead endpoint read as available, so point modelRoles.speech at such a provider only after it has answered a real synthesis.

A gateway's own /api/v1/audio/speech route is NOT an equivalent: OpenWebUI substitutes its own voice enum before forwarding, so a cloned pack name comes back 400 from the upstream. Keep modelRoles.speech on the direct endpoint.

Three speech-role laws (bundle-read + measured 2026-10-18)

  1. No setting carries a voice for openai-speech. The transport builds the body with voice from the call's voice_id and nothing else. tts.localVoice is the local Kokoro backend's key; speech.voice is the TUI's speak-aloud voice, an enum of Kokoro names that never reaches an HTTP request (setting it to a pack name breaks the vocalizer and does nothing for synthesis — read cfgSpeechVoice in dist/cli.js before believing otherwise). So a voice-less call is the harness's normal shape: a voice-required endpoint needs a server-side default (fleet answer: DEFAULT_VOICE in scripts/frank-tts.env, journal marks the line (default), unknown names still 400 with the list).
  2. An explicit role has no fallback candidate. modelRoles.speech set to one model yields exactly one candidate, so a dead endpoint or an unresolvable selector fails with "No available speech model matches the speech role" instead of degrading to Kokoro. Loud-by-design; do not "fix" it by adding a fallback in the client.
  3. models.yml loads at process start; config.yml is read live. A session older than the new row fails exactly like a dead endpoint while a fresh process succeeds in the same minute.

Diagnosing "speech device won't speak" — in this order

  1. curl the endpoint directly with a real voice name. 200 + audio ⇒ the service is fine and the fault is harness-side; stop touching the server.
  2. Control test in the same session: point modelRoles.speech at local/kokoro and call the device. It works ⇒ config.yml is live-read and the runner selector or its voice argument is the problem, not the endpoint. Revert the role immediately after.
  3. Distinguish stale registry from dead endpoint with a fresh process: omp -p --no-session --no-extensions --no-skills --no-rules --tools tts --cwd /tmp 'Call the tts tool exactly once with text="…" output_path=/tmp/x.wav voice_id=<name> — then reply with the tool result line.' Succeeds ⇒ the long-lived session predates the models.yml row; restart it.
  4. Cross-check server-side, never trust the client's echo: the façade/access log line and the engine's own ledger must carry the same byte count the tool printed. (model=… in a tool result is what the client requested; a server line is the fact.)
  5. For harness-behaviour claims, read the bundle rather than reasoning from the tool description: /home/blank/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js, e.g. grep -o -E '.{300}response_format:[a-zA-Z$]+.{250}' for the speech body builder, grep -o -E 'cfgSpeechVoice:.{0,400}' for what a settings key feeds.