serverAppConfigs/serverAppConfigs-docs/Dankclaw/OpenWebUI.md

type, description, tags, timestamp

type	description	tags	timestamp
reference	OpenWebUI is a self-hosted chat interface for LLMs.	dankclaw openwebui	2026-09-21

OpenWebUI

OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compatible endpoints (vLLM, llama.cpp, Ollama) and adds RAG, plugins, tool calling, and multi-user support.

Host: Dankclaw — Port: 3005 (host) → 8080 (container) — Vhost: ai.franksdank.cloud routes straight to 192.168.0.143:3005 (unguarded — OpenWebUI enforces its own auth) — Boot-persistent: Quadlet — Docs: https://github.com/open-webui/open-webui

Facts

Item	Value
Image	ghcr.io/open-webui/open-webui:main (verified from the live quadlet unit)
Upstream latest	0.11.4 (2026-09-21) — security/access-control advisory; :main is the rolling image, not a stable pin. Imagegen endpoint is POST /api/v1/images/generations (the comfyui engine is still upstream).
Port	3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable.
Data volume	/home/dank/Containers/open-webui/data → /app/backend/data
Backends	Picker exposes exactly one model — frank-zealot (host.containers.internal:7771). The frank-dragoon and frank-archon connections remain configured in the DB, disabled (2026-10-07) — see The LLM backends
Network	Pinned bridge systemd-open-webui (10.89.3.0/24, gateway 10.89.3.1) — see History
Persistence	Quadlet unit open-webui.service; [Install] WantedBy=default.target
Deploy	scripts/deploy-open-webui-quadlet.sh dankclaw

The LLM backends

The picker is built from the model table in webui.db (one row per model) plus the model_ids of each enabled OpenAI connection, NOT from live connection discovery — so the picker shows exactly what is seeded there. As of 2026-10-07 the picker exposes exactly one model: frank-zealot.

Connection (openai.api_configs idx)	URL	Enabled	model_ids	Runtime	Host
0	http://host.containers.internal:7770/v1	no (2026-10-07)	frank-dragoon	halogen flash-server (Strix Halo iGPU)	Dankclaw
1	http://192.168.0.235:7770/v1	no (2026-10-07)	frank-archon	radiance TP=2 (2× R9700)	Dankserver
2	http://host.containers.internal:7771/v1	yes	frank-zealot	llama.cpp CUDA (RTX 3090, 2×200,704)	Dankclaw

The model table carries exactly one decoration row, frank-zealot (context length 200,704 = the per-slot window; description: Qwen3.5-9B Q6_K + BF16 vision mmproj).

A third gate sits after the table, and it is the one that bites family accounts (2026-10-07): a model row is only visible to a non-admin user when its access permission grants them. A seeded row defaults to admin-only, so the picker renders "No results found" for every regular user even though the connection is enabled, model_ids is correct, and the endpoint answers /v1/models — and GET /api/v1/models returns [] for that user while an admin token returns the model. Fix: set the row's access_control to public (the access column on the model row) via the admin UI (Admin Panel → Models → visibility) or the DB. The verification battery below must therefore run as a non-admin user, not only as an admin.

Two gotchas make the wiring non-obvious:

• llama.cpp /v1/models is Ollama-shaped. llama.cpp returns {"models":[{"name":...}]}; OpenWebUI's extract_data() only reads the OpenAI {"data":[{"id":...}]} shape, so connection discovery silently misses the llama.cpp endpoints (frank-dragoon, frank-zealot). The fix is to set model_ids in each connection's api_config — when non-empty, OpenWebUI builds the model list directly and never fetches /v1/models. frank-medium (vLLM) serves the OpenAI shape and would discover fine, but uses model_ids too for consistency.
• The host's own services are reached via host.containers.internal. The container sits on a netavark bridge; it can hairpin to a different host's LAN IP (dankserver 192.168.0.235) but not to its own host's non-published ports via the host LAN IP (192.168.0.143:7770/7771 is refused). host.containers.internal (169.254.1.2) resolves to the host on the bridge and reaches both llama.cpp services.

The Ollama connection is disabled (ollama.enable=false).

All three endpoints are bare-metal systemd services managed by this repository. Model swaps, context sizing, and spec tuning happen there. See the LLM knowledge base. Disabling a connection here only hides it from this picker — the endpoint itself keeps serving every other client (the harness, Hermes, API consumers).

Container introspection (DB and backend python)

• Working dir for imports: /app/backend; package name open_webui (underscore), NOT openwebui. os.environ["WEBUI_SECRET_KEY"] MUST be set BEFORE importing any open_webui module (import-time hard requirement); the secret lives at /app/backend/.webui_secret_key.

• The sqlite3 CLI is NOT in the container — use Python's sqlite3 module (podman exec open-webui python3 -c "import sqlite3; conn = sqlite3.connect('/app/backend/data/webui.db'); …"). For config keys use parameterized LIKE queries (avoids shell quoting).

• Backend python: podman exec -w /app/backend open-webui python3 -c "…" — Models.get_all_models() is a coroutine (await it); model objects are pydantic ModelModel instances (attribute access, not .get()), with NO status/enabled attribute — use is_active from the DB.

• API auth for verification: create_token lives in open_webui.utils.auth (NOT open_webui.utils.utils); get_verified_user requires an id claim (not just sub); SESSION_SECRET = WEBUI_SECRET_KEY. Mint an admin token inside the container:

  podman exec -i open-webui sh -c 'export WEBUI_SECRET_KEY=$(cat /app/backend/.webui_secret_key); \
    python -c "from open_webui.utils.auth import create_token; \
    print(create_token({\"id\":\"<UID>\",\"sub\":\"<UID>\",\"email\":\"…\",\"role\":\"admin\"}))"'
  

  (admin UID from the user table).

Verification battery

1. GET /api/v1/models/list?limit=100 → returns EXACTLY the intended models. Response key is items, not data.
2. POST /api/chat/completions per model with a trivial prompt ("Reply with exactly one word: pong") → each returns content pong. Reasoning models need a generous budget: frank-zealot (Qwen3.5 template) returns empty content + finish_reason: "length" at max_tokens 200 (the thinking block eats it) and answers pong at 1500.
3. A 500 "Server Connection Error" → podman logs open-webui for ClientConnectorError: a network-reachability problem (host LAN IP vs host.containers.internal), not a model problem.

Durability: the model table + connections are DB-only (not git-managed); a fresh data volume loses them — including the public access grant above. For reproducibility, add an idempotent first-start seed script or document the setup in the service doc.

API access (auth-gated)

Every /api/* endpoint requires an authenticated session (SSO login JWT) or a per-user API key (auth.enable_api_keys=true; keys are minted per account under Profile → API Keys). Anonymous web visitors get the pending role (ui.default_user_role=pending), which cannot chat. Verified 2026-10-07 from loopback :3005 and over the public vhost: GET /api/v1/models/list, GET /api/v1/models and POST /api/chat/completions all return 401 with no key (also with a bogus bearer token); only /api/config, /api/version and /health answer publicly, and none of them expose the model list. An external program needs an account + API key — there is no keyless access.

Web search

Chat web search is served by SearXNG on this host, wired in the quadlet (ENABLE_WEB_SEARCH, WEB_SEARCH_ENGINE=searxng, SEARXNG_QUERY_URL).

• Access path: the backend calls http://192.168.0.143:8081/search?format=json directly — gluetun's published port, NOT the search.franksdank.cloud vhost. The Anubis→Authentik guard chain is a browser mechanism (proof-of-work + session flow); a plain HTTP client cannot solve it, and the internal port bypasses Caddy entirely.
• Limiter: SearXNG caps non-HTML (JSON) requests at 4/IP/hour (API_MAX, a hardcoded Python constant — no config knob). The pass_ip list is checked FIRST and short-circuits the whole botdetection chain, and OpenWebUI's bridge egress (10.89.x) is in pass_ip, so the cap never applies. Verified: an 8-request burst from the container returned 8×200.
• Config storage: OpenWebUI keeps web-search settings per-key in its SQLite config table (webui.db, web.search.*). The quadlet env vars seed that table on first start (fill-missing only); afterwards the DB is authoritative and Admin-UI edits persist there.

SSO (authentik)

Login is authentik via the OIDC authorization-code flow (wired 2026-09-24). The vhost stays exempt from the Anubis guard chain: the OAuth dance is browser-driven and only touches the two exempt vhosts (ai. and auth.).

• authentik objects (created via the admin API, 2026-09-24): application open-webui (slug open-webui) with the OAuth2 provider Open WebUI (confidential client, authorization_code + refresh_token, Strict redirect URI https://ai.franksdank.cloud/oauth/oidc/callback, implicit consent flow). The client ID is in the quadlet; the client secret lives in the git-ignored host file /home/dank/Containers/open-webui/env (mode 600, loaded via EnvironmentFile=).
• Roles: the scope mapping Open WebUI roles (scope roles) reads the application entitlements Open WebUI Users → user and Open WebUI Admins → admin. The entitlements are bound to the groups openwebui_users / openwebui_admins. akadmin (frank8465@protonmail.com) is in openwebui_admins; qa-e2e-admin is in openwebui_users.
• Account merge: OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true folds an SSO login into the existing local account with the same email — the pre-SSO admin (frank8465@protonmail.com) keeps its data. ENABLE_OAUTH_SIGNUP=true lets other openwebui_users create accounts on first login.
• First login mints role pending. ui.default_user_role=pending (config DB, verified 2026-10-09), so a freshly-provisioned openwebui_users account completes SSO and then cannot chat (picker empty) until its role is flipped: UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud'; — run with host python3 over ssh against /home/dank/Containers/open-webui/data/webui.db (no sqlite3 CLI on Dankclaw; table is user, singular; no active column). No per-user model grant is needed: the picker model frank-zealot carries the global access_grant row (model, frank-zealot, user, *, read), so any role='user' account sees it. Flip on FIRST-LOGIN SIGHTING — the row does not exist until the account signs in (2026-10-09: five provisioned accounts had no row).
• IdP reachability: the container is on its own bridge (10.89.3.0/24) and cannot reach authentik-server:9000 by name. It reaches the IdP over the public URL https://auth.franksdank.cloud — Caddy publishes 0.0.0.0:443 (netavark DNATs published ports from other containers) and Pi-hole resolves the name to the host LAN IP. The server-side code exchange rides that path.
• Config authority: the OAuth env vars in the quadlet seed the per-key config DB on first run (fill-missing only); on the live data dir the Admin Panel is authoritative. To rotate the client secret: create a new token in authentik, update the host env file, and set the new value in the Admin Panel (or the DB), then restart the container.
• SSO-only (2026-09-24): after the OIDC round-trip was verified, the local username/password path was retired. ENABLE_LOGIN_FORM=false hides the form and ENABLE_PASSWORD_AUTH=false refuses password logins, so the only way in is the authentik OIDC flow. The ui.enable_login_form config-DB override was flipped to false in the same change — the per-key DB wins over env, so both must agree, and the DB value is the one to change on a live data dir. To restore the password form: set ui.enable_login_form back to true in the Admin Panel (or DB) and drop the two env vars from the quadlet.

Boot persistence (Quadlet)

The unit lives at dankclaw/open-webui/quadlet/open-webui.container. On the host it installs to ~/.config/containers/systemd/open-webui.container. The generator turns it into open-webui.service. Because the unit sets [Install] WantedBy=default.target and user linger is on, it starts at boot. systemctl --user is-enabled open-webui.service reports generated (the quadlet boot-start state).

Volumes

Container path	Host path
/app/backend/data	/home/dank/Containers/open-webui/data

The data dir holds the database, RAG indexes, and uploads. The deploy script creates it if missing and preserves it across reruns.

Healthcheck

The quadlet carries a built-in healthcheck (30s interval, 3 retries, 300s start period). It checks two things from inside the container:

1. App liveness — GET http://127.0.0.1:8080/health returns 200 ({"status":true}).
2. Network liveness — a non-loopback interface exists in /sys/class/net. The dead-tap zombie state (History) leaves only lo behind, and this is the check that catches it.

podman ps shows the container as unhealthy if either fails. Probe manually at http://dankclaw:3005/health.

Image generation (ComfyUI)

Image generation rides the local ComfyUI integration, not the chat picker — ComfyUI's models never appear in the chat model picker (that is chat LLMs only), so ComfyUI needs its own image-generation connection configured in OpenWebUI. ComfyUI itself runs GPU-enabled on Dankclaw and serves :8188.

Endpoint: POST /api/images/generations (image 0.11.x: upstream moved the route to POST /api/v1/images/generations; the comfyui engine is still upstream) with Authorization: Bearer <token> and body {"model":"imagegen","prompt":"…","n":1,"size":"1024x1024"}; the response carries base64 image data or a URL.

Failure modes: connection refused ⇒ OpenWebUI or ComfyUI is down; timeouts on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.

Retired instances

• The compose open-webui service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
• The bare-container open-webui on Dankserver (port 3000) pointed at a dead backend (:9090) and was removed. Its data dir is preserved on Dankserver in case it is needed.
• The bare-container searxng on Dankserver (port 8081) was removed. SearXNG now runs only on Dankclaw behind Caddy.

Gotchas

• The image tracks upstream main; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see ComfyUI.)
• The default admin account is created on first login; secure it immediately.
• If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
• The on-host LLM services (Dankclaw :7770, :7771) are reached via host.containers.internal, NOT the host LAN IP: the container's netavark bridge cannot hairpin to its own host's non-published ports via 192.168.0.143 (connection refused), and the bridge gateway (10.89.3.1) lives in the rootless netns while llama-server runs in the host netns, so it refuses too. host.containers.internal (169.254.1.2) resolves to the host on the bridge and works — verified end-to-end for both endpoints. The Dankserver backend is a different host, reached via its LAN IP 192.168.0.235:7770.

History

• 2026-08-22 — slirp4netns → pinned bridge. The unit ran on Network=slirp4netns. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only lo in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.

• 2026-09-21 — model picker rebuilt for the 3-model fleet. The picker carried 39 stale rows (37 dead Ollama models + 2 frank-default-*) and was missing frank-medium. Rebuilt the model table to exactly frank-dragoon/frank-medium/frank-zealot and set model_ids on each connection (llama.cpp's Ollama-shaped /v1/models is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at host.containers.internal — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via /openai/chat/completions.

• 2026-09-24 — SSO via authentik. OIDC authorization-code flow wired per the SSO (authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the roles scope claim. E2E coverage in tests/e2e/specs/18-openwebui-sso.spec.ts.

• 2026-09-24 — flipped to SSO-only. The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: ENABLE_LOGIN_FORM=false + ENABLE_PASSWORD_AUTH=false in the quadlet and ui.enable_login_form set to false in the config DB (the DB wins over env, so both were changed together). qa-e2e-admin was demoted out of the authentik Admins group (it keeps openwebui_users / homepage-admins for app access).

• 2026-10-07 — picker narrowed to frank-zealot only. Per Frank: the only model accessible in OpenWebUI is frank-zealot. Rebuilt the model table to a single frank-zealot row (the old rows frank-dragoon/frank-archon/frank-small are deleted), set openai.api_configs indices 0 (frank-dragoon) and 1 (frank-archon) to enable: false, and retargeted index 2's model_ids from the stale frank-small label to frank-zealot — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: /home/dank/Containers/open-webui/data/webui.db.bak-20261007 (+ -wal/-shm). Verified: /api/v1/models/list returns exactly ["frank-zealot"], chat completion answers pong (needs max_tokens≥1500 — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see API access).

• 2026-10-09 — re-keyed to the Protoss ids (wire rename). The endpoints began serving frank-zealot / frank-dragoon / frank-archon, so three id-keyed objects had to move in the same window as the alias flip: model.id+name (frank-zealot → frank-zealot, params {"context":{"length":200704}} and the meta.description kept), access_grant (resource_type='model', resource_id, principal user/*, permission read), and openai.api_configs per-backend model_ids (index 2 = :7771 is the enabled family backend; indices 0/1 = :7770 and Dankserver :7770 stay enable: false). Snapshot first: /home/dank/Containers/open-webui/data/webui.db.bak-20261009. Apply from inside the container (podman exec -i open-webui python3 -, sqlite3; the image has no sqlite3 binary) then systemctl --user restart open-webui.service (it is a rootless quadlet) to drop the cached model list. Verify in a real browser as the family QA account: the welcome screen selects frank-zealot and answers. A stale id fails silently — the picker simply lists nothing, because OpenWebUI intersects the backend's /v1/models with model_ids.

  Law: the model id is a foreign key in three tables. Any future alias change on an endpoint OpenWebUI fronts must ship with this re-key, in the same deploy.

• 2026-10-09 — composer is contenteditable, not <textarea>. A Playwright probe that waits for textarea after SSO times out on the welcome screen: OpenWebUI 0.6 renders the composer as a contenteditable div. The stable anchor is the model button (getByRole("button", { name: /selected model/i })) and the welcome text "How can I help you today?"; that snapshot is also the cheapest proof the picker is populated.

See Also

• Dankclaw Stack
• Hermes — the agent OpenWebUI can talk to
• SearXNG — the search engine on this host
• LLM knowledge base — the vLLM and llama.cpp endpoints