From f3360a28f1e20853274ff24bbb535d78b88ff5ea Mon Sep 17 00:00:00 2001 From: Frank Date: Wed, 21 Oct 2026 16:48:29 -0700 Subject: [PATCH] feat(models): reach the fleet from off-network through the OpenWebUI door MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every provider in agent/models.yml is a LAN address, so a workstation that leaves the house has no model at all. Add `openwebui (franksdank.cloud)`: the already-public OpenWebUI vhost as an OpenAI-compatible provider that proxies the same engines, with frank-dragoon and frank-zealot rows and the WAN row as its own compaction target. LAN rows, modelRoles and the speech runner stay untouched — the door is purely additive. The key is command-backed (`apiKey: "!cat ~/.config/secrets/openwebui_api_key"`, mode 600) instead of inline: a literal sk-... in a tracked file is a key in every agent context on every box that pulls this repo, and the pre-commit shape gate would block it anyway. Rotate = rewrite the file + restart omp, because a `!cmd` value is cached for the process lifetime. Compat blocks copy their LAN twins because the proxy was probed, not assumed (2026-10-21): streaming SSE carries reasoning_content and tool_calls deltas plus final usage, image_url content parts reach both vision towers, and chat_template_kwargs is forwarded — proven on frank-zealot, where enable_thinking false yields zero reasoning and true a real block. frank-archon gets no WAN row: OpenWebUI answers 400 "Model not found" because its picker never published it, so that path is a fleet change. Verified end-to-end from a scratch HOME whose LAN baseUrls point at a dead port, so any answer can only have come through the door (text and a real tool round-trip); `omp models` still lists all four chat providers and the tts runner, so the file validates. bats 137/137, kb-lint clean. Docs: omp-docs/core/wan-model-access.md. --- agent/models.yml | 132 ++++++++++++++++++++++++++++ omp-docs/core/index.md | 1 + omp-docs/core/wan-model-access.md | 139 ++++++++++++++++++++++++++++++ omp-docs/log.md | 18 ++++ 4 files changed, 290 insertions(+) create mode 100644 omp-docs/core/wan-model-access.md diff --git a/agent/models.yml b/agent/models.yml index 37cd2c3..f768f44 100644 --- a/agent/models.yml +++ b/agent/models.yml @@ -306,3 +306,135 @@ providers: name: "frank-tts (dankclaw) — cloned voices" api: openai-speech input: [text] + + # ─── WAN door ─────────────────────────────────────────────────────────── + # The only block in this file that answers OFF the home network. Every + # provider above is a LAN IP, and `modelRoles` (config.yml) still points at + # those, so away from home you must NAME one of the rows below: + # omp --model 'openwebui (franksdank.cloud)/frank-dragoon:medium' + # The door is the OpenWebUI vhost `ai.franksdank.cloud` — Caddy → OWU on + # Dankclaw:3005, Anubis-exempt because OWU enforces its own auth, public A + # record present (LAN clients resolve 192.168.0.143 via Pi-hole, measured + # 2026-10-21). It exposes an OpenAI-compatible surface at /api/v1 and + # proxies to the SAME engines as the LAN rows — no second inference stack, + # so every engine-side law below still holds through it. + # CREDENTIAL LAW: the key is not in this file. `apiKey` accepts a + # command-backed value (`!cmd`, resolved by src/config/resolve-config-value.ts: + # stdout trimmed, cached for the process lifetime; anything else is tried as + # an env-var name, then taken literally), so the secret lives in + # ~/.config/secrets/openwebui_api_key, mode 600, like every other credential + # on these boxes. A literal `sk-…` here would trip the pre-commit shape gate + # and then live in every agent context on every box that pulls this repo. + # Rotate: rewrite the secret file, then RESTART omp — the `!cmd` result is + # cached per process, so a live session keeps serving the old key. + # WHAT THE DOOR SHOWS (probed 2026-10-21 with this key): /api/v1/models + # answers frank-dragoon and frank-zealot only. frank-archon is NOT published + # in OpenWebUI's picker and the API answers 400 "Model not found" for it, so + # the fleet's biggest endpoint has no WAN path until it is published + # server-side on Dankclaw — a fleet change, not a harness change. + # CONTRACT PROBED THROUGH THE DOOR (2026-10-21): streaming SSE carries + # `reasoning_content` deltas and `tool_calls` deltas, finish_reason + # `tool_calls`, and a final `usage` under stream_options.include_usage; + # `image_url` content parts reach both vision towers (a 64 px red PNG comes + # back "Red" from each); and `chat_template_kwargs` IS forwarded — proven on + # frank-zealot, where enable_thinking false gives reasoning 0 chars and true + # gives a real thinking block on the same prompt. Every compat block below is + # therefore a copy of its LAN twin, never a simplification: dropping the + # kwargs here would quietly change thinking behaviour on the WAN path. + # Chat completions made with an API key create NO OpenWebUI chat row (the + # chat list is unchanged after probing), so an off-network session leaves no + # trail in the family's chat history. + # NOT BEHIND THIS DOOR: speech. OWU does route /api/v1/audio/speech and it + # does reach frank-tts:7772, but OWU substitutes its own voice enum first — + # a cloned pack name comes back 400 from the upstream. `modelRoles.speech` + # stays on `frank-tts (dankclaw)`; do not repoint it here. + "openwebui (franksdank.cloud)": + baseUrl: https://ai.franksdank.cloud/api/v1 + api: openai-completions + # Quoted because an unquoted leading `!` is a YAML tag, not a command. + apiKey: "!cat ~/.config/secrets/openwebui_api_key" + models: + - id: frank-dragoon + # Same halogen flash-server as `llamacpp (dankclaw)` above, one hop + # further out; the picker row and the engine share this id, so there + # is no alias to keep in sync. + name: "frank-dragoon (WAN)" + # Off-network the compaction hub must be reachable too: the LAN + # frank-zealot row is a dead address from anywhere but the house. + # Same engine, same slot arithmetic, so the keepRecent/threshold + # math in the LAN rows transfers untouched. + compactionModel: openwebui (franksdank.cloud)/frank-zealot + # Inherited from the engine, not invented: 200,000 under the native + # 262,144 ceiling (the 400-zone law — never sit at the ceiling). The + # proxy neither widens the window nor truncates: an over-ceiling + # request is still a 400. + contextWindow: 200000 + # Same clamp arithmetic as the LAN row (see frank_dragoon_unit.bats); + # HALOGEN_MAX_TOKENS_CAP on the engine is unchanged. + maxTokens: 45056 + reasoning: true + input: [text, image] + compat: + # Verified through the door: reasoning arrives as + # `reasoning_content` in the deltas, not folded into content. + reasoningContentField: "reasoning_content" + # `:off` must be a real switch, and it is: this engine honours + # reasoning_effort "none"; the harness default would only send the + # lowest effort, which still thinks. + reasoningDisableMode: none-effort + # Identity map, same reason as the LAN row — flash-server lists + # minimal|low|medium|high|xhigh|max|none and 400s anything else. + reasoningEffortMap: + minimal: minimal + low: low + medium: medium + high: high + xhigh: xhigh + max: max + # Pinned so a server-side HALOGEN_TOP_K edit cannot change this + # session's recipe; the engine accepts all three keys. + extraBody: + temperature: 1.0 + top_p: 0.95 + top_k: 20 + - id: frank-zealot + # The 3090 endpoint, same llama.cpp engine behind the proxy. Stays + # the widest slot in the file, so it is the WAN compaction target. + name: "frank-zealot (WAN)" + compactionModel: openwebui (franksdank.cloud)/frank-zealot + # PER-SLOT window: llama.cpp divides --ctx-size across slots and + # answers a longer prompt with a 400, never a truncation. Follows + # every reslice on the engine (3 × 200,704 today). + contextWindow: 200704 + # Prompt + thinking budget inside the slot; compaction fires at 85 % + # of the window on this row set (see the LAN twin for the full + # worst-case arithmetic). + maxTokens: 16384 + reasoning: true + input: [text, image] + compat: + reasoningContentField: "reasoning_content" + # LOAD-BEARING, exactly as on the LAN row: the Bonsai-2 template + # accepts ONLY xhigh|medium|low and raises on anything else, so an + # unmapped harness level is a hard failure on the role's first + # request. Probed 2026-10-21 that the mapped value, not the + # harness-level, is what reaches the template through this proxy. + reasoningEffortMap: + minimal: low + low: low + medium: medium + high: xhigh + max: xhigh + # The payload's own sampling recipe plus the thinking switches. + # chat_template_kwargs survives the proxy (verified), so + # preserve_thinking keeps prior assistants' thinking blocks in + # history for multi-turn tool traffic — same contract as at home. + extraBody: + temperature: 1.0 + top_p: 0.95 + top_k: 20 + min_p: 0.0 + presence_penalty: 0.0 + chat_template_kwargs: + enable_thinking: true + preserve_thinking: true diff --git a/omp-docs/core/index.md b/omp-docs/core/index.md index b6dfe70..7b90f44 100644 --- a/omp-docs/core/index.md +++ b/omp-docs/core/index.md @@ -15,5 +15,6 @@ These files define the structural conventions that govern the agent system itsel * [dankvoice](dankvoice.md) — Voice output is the `dank/dankvoice` package (`pi install git:`); this page keeps fleet wiring and the cutover procedure. * [OMP Status Line Customization](statusline.md) — How presets, composer shapes, and segment arrays actually interact; the fleet's tuned bar. (moved from /pi/, 2026-10-19) * [OMP Subagent Limits & Fan-out Policy](subagent-limits.md) — Live caps: concurrency 2, no nested spawns (the `maxRecursionDepth: 0` trap), per-endpoint in-flight ceilings, isolation policy. (moved from /pi/, 2026-10-19) +* [WAN Model Access](wan-model-access.md) — Driving the fleet's models from outside the home network through the OpenWebUI door: the command, the key file and its rotation law, what the door publishes, and the off-network failure modes. **Change log:** the bundle keeps exactly one, one level up — [Bundle Change Log](../log.md). A directory here never has its own. diff --git a/omp-docs/core/wan-model-access.md b/omp-docs/core/wan-model-access.md new file mode 100644 index 0000000..e8fc1d5 --- /dev/null +++ b/omp-docs/core/wan-model-access.md @@ -0,0 +1,139 @@ +--- +type: reference +title: WAN Model Access (ai.franksdank.cloud) +description: How to drive the fleet's models from outside the home network — the OpenWebUI door in agent/models.yml, the key file and its rotation, what the door does and does not publish, and the off-network failure modes. +tags: + - omp + - models-yml + - openwebui + - remote-access + - fleet +verified: 2026-10-21 +timestamp: 2026-10-21 +--- + +# WAN Model Access (ai.franksdank.cloud) + +Every provider in `agent/models.yml` except one is a LAN address, so a laptop +that leaves the house loses the fleet. The exception is the +`openwebui (franksdank.cloud)` block: the fleet's already-public OpenWebUI vhost +speaks OpenAI-compatible HTTP at `/api/v1` and proxies to the same engines the +LAN rows point at. One hop further out, no second inference stack. + +Read this page when you are off-network and a model call fails, when the key +needs rotating, or before adding another row behind the door. + +## Run a session away from home + +The door is added as a provider, never as a role default. `modelRoles` in +`agent/config.yml` still names LAN providers, so away from home you must name +the model explicitly: + +```bash +omp --model 'openwebui (franksdank.cloud)/frank-dragoon:medium' +omp --model 'openwebui (franksdank.cloud)/frank-zealot:off' +``` + +The same two ids work in the `/model` picker inside a running session — the +provider label is what disambiguates them from the LAN rows, since the wire ids +are identical by design (OpenWebUI's picker row and the engine share the id, so +there is no alias to keep in sync). + +## The key + +`apiKey: "!cat ~/.config/secrets/openwebui_api_key"` — a **command-backed** +value, resolved by `src/config/resolve-config-value.ts` in the harness: stdout +is trimmed, an env-var name is tried next, and only then is the string taken +literally. The secret itself is a 600-mode file in `~/.config/secrets/`, the +same convention as `gitea` and `kagi_api_key`. + +Laws: + +- **Never inline the `sk-…`.** It is a tracked file; the pre-commit shape gate + blocks `sk-…` anywhere in the staged diff, and a key that slipped through + would live in every agent's context on every box that pulls this repo. +- **Rotation is: rewrite the secret file, then restart omp.** The `!cmd` result + is cached for the process lifetime, so a long-lived session keeps serving the + old key — a stale-session failure reads exactly like a revoked key. +- The key is an **OpenWebUI** key (User menu → Settings → Account → API keys), + not a fleet endpoint key. It carries that user's model ACL, so what the door + shows is whatever the picker shows that user. +- Every other workstation needs its own copy of the file; nothing in the repo + carries the value. + +## What the door publishes + +Probed 2026-10-21 with the fleet key. + +|Behind the door|Notes| +|---|---| +|`frank-dragoon`|halogen flash-server, Dankclaw iGPU — same row as the LAN `llamacpp (dankclaw)`| +|`frank-zealot`|llama.cpp on Dankclaw's 3090 — same row as `llamacpp (dankclaw-3090)`; also the WAN compaction target| +|`frank-archon`|**not published.** `/api/v1/models` omits it and the API answers `400 "Model not found"`. Dankserver's endpoint has no public face at all. Giving it a WAN path is a fleet change on Dankclaw (publish it in the OpenWebUI picker), not a harness change| +|`frank-tts`|route exists, contract does not. OpenWebUI forwards `/api/v1/audio/speech` to `frank-tts:7772` but substitutes its own voice enum first, and a cloned-pack name comes back `400` from the upstream. `modelRoles.speech` stays on the LAN `frank-tts (dankclaw)` row| + +## Contract verified through the proxy + +Each of these was measured end-to-end on 2026-10-21 — they are why the compat +blocks in the WAN rows are copies of their LAN twins rather than simplifications: + +- Streaming SSE carries `reasoning_content` **and** `tool_calls` deltas, + `finish_reason: tool_calls`, and a final `usage` under + `stream_options.include_usage`. +- `image_url` content parts reach both vision towers. +- `chat_template_kwargs` is forwarded — proven on `frank-zealot`, where + `enable_thinking: false` yields zero reasoning and `true` yields a real + thinking block on the same prompt. Drop the kwargs from a WAN row and you + silently change thinking behaviour on the WAN path. +- API-key completions create **no** OpenWebUI chat row: the chat list is + unchanged after a probe, so an off-network session leaves nothing in the + family's chat history. + +Context windows, `maxTokens`, sampling and effort maps are inherited from the +engines and documented on the LAN rows and in the fleet vault +(`serverAppConfigs-docs/LLM/`). The proxy neither widens a window nor truncates: +an over-ceiling request is still a `400`. + +## Off-network failure modes + +- **`Connection refused` / long timeout on a LAN id.** Not a service outage — + you are not home. Switch to the WAN id. +- **`Model not found` for `frank-archon`.** Expected; see the table above. +- **Compaction wants a LAN address.** Each WAN row sets `compactionModel` to + the WAN `frank-zealot`, but `compaction.remoteEndpoint` in `agent/config.yml` + is a LAN URL. It is last in `methodOrder`, so the other methods run first and + the remote attempt just fails — leave it unless off-network compaction turns + out to need it. +- **Roles still resolve to LAN.** A full off-network session that must not + depend on hand-picked models needs `retry.fallbackChains` entries pointing + the LAN defaults at the WAN rows; that is a deliberate `config.yml` change, + not part of adding rows here. + +## Verify the door + +```bash +# Key + model list (LAN clients resolve 192.168.0.143 via Pi-hole; the WAN +# A record is what makes this reachable from elsewhere). +curl -s https://ai.franksdank.cloud/api/v1/models \ + -H "Authorization: Bearer $(cat ~/.config/secrets/openwebui_api_key)" | jq -r '.data[].id' + +# Registry loaded, WAN rows present, no validation failure (a failed load +# silently disables EVERY custom provider — see +# agent/models.yml's schema-law comment): +omp models --no-extensions | grep -E 'franksdank|validation failed' +``` + +A scratch-HOME run is the only real proof that a harness change reaches the +door: copy `models.yml` + `config.yml` to `$TMP/.omp/agent/`, symlink +`$TMP/.config/secrets` at the real one (a `~` in a `!cmd` value expands to +`$HOME`), rewrite the LAN baseUrls to a dead port so a success cannot be +faked, then `HOME=$TMP omp --model 'openwebui (franksdank.cloud)/…' -p …`. +Delete the scratch tree afterwards — it holds endpoint URLs. + +## Pointers + +- Door topology: [Caddy](/caddy/index.md), [System hosts](/system/index.md). +- Model registry law and per-provider limits: [Core config](index.md), + [OMP Subagent Limits](subagent-limits.md). +- Server-side record: `serverAppConfigs-docs/Dankclaw/OpenWebUI.md` + (that repo's vault, not this bundle). diff --git a/omp-docs/log.md b/omp-docs/log.md index 31cf628..50f5cfd 100644 --- a/omp-docs/log.md +++ b/omp-docs/log.md @@ -24,6 +24,24 @@ long form. ## 2026-10-21 +### The WAN door into the model registry + +`openwebui (franksdank.cloud)` joins `agent/models.yml` — the already-public +OpenWebUI vhost as an OpenAI-compatible provider, so a workstation outside the +house still reaches `frank-dragoon` and `frank-zealot` while every LAN row stays +untouched. The credential is command-backed +(`apiKey: "!cat ~/.config/secrets/openwebui_api_key"`, 600-mode file): a literal +`sk-…` in a tracked file is a key in every agent context on every box that pulls. + +Laws: **OpenWebUI forwards `chat_template_kwargs`**, so a proxied row copies its +LAN twin's compat block instead of simplifying it — proved on `frank-zealot`, +`enable_thinking` false giving zero reasoning and true a real block on one +prompt; **the picker IS the API's model list**, `frank-archon` answers `400 +"Model not found"` because it was never published, so a WAN path for it is a +fleet change on Dankclaw and not a harness one; and **a `!cmd` value is cached +for the process lifetime**, so rotating a key without restarting omp leaves the +old key working. Page: [core/wan-model-access.md](core/wan-model-access.md). + ### the audit grows an advisory table, and corrects three of its own numbers `omp-docs/reports/fleet-security-audit-2026-10-21.html` folds in the two per-host vulnerability