Files

T

Frank c353725ea5 docs: cross-link the vllmScripts project (model endpoints + git host)

- README: fix broken 'Also See' placeholder URL -> live Gitea LAN URL
- index.md: expand vllmScripts See Also with the endpoints it serves
- OpenWebUI.md: :7770 backend is the llama.cpp PP head (split to the
  3090), managed by vllmScripts; note the FLM NPU :52625 alternative;
  fix the broken [[llama.cpp Server]] wikilink
- Gitea.md: hosted repositories table (both projects)
- Dankclaw Stack.md: host-level LLM endpoints in the service inventory
- AGENTS.md: runtime note — LLM endpoints live in vllmScripts, not here

Pre-existing failures (untracked ComfyUI WIP): tests 6, 8, 9, 10.
Docs seams green: ok 19 (compose service docs), ok 20 (index links).

2026-08-16 23:19:54 -07:00

2.4 KiB

Raw Blame History

OpenWebUI

OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compatible endpoints, Ollama, llama.cpp, and vLLM, and adds RAG, plugins, tool calling, and multi-user support.

Host: Dankclaw — Hostname: chat.dankclaw.local — Port: 3000 — Docs: https://github.com/open-webui/open-webui

Facts (from research, primary source)

Item	Value
Image	`ghcr.io/open-webui/open-webui:main` (CPU build for non-NVIDIA hosts)
Port	3000 (host) → 8080 (container)
Data volume	config, sessions, RAG indexes
Backend	OpenAI-compatible endpoint at `http://localhost:7770/v1` — the llama.cpp PP head (Qwen3.8-27B, tensors split to the Dankserver RTX 3090 worker)

Note: the :cuda image tag was previously used on Dankserver. Dankclaw is AMD; use the default (CPU) image, NOT the cuda tag.

Connecting to llama.cpp

OpenWebUI talks to the local llama.cpp server through its OpenAI-compatible API. Configure a new connection in the UI or set the base URL:

environment:
  OPENAI_API_BASE_URL: "http://host.containers.internal:7770/v1"
  OPENAI_API_KEY: "none"

The hostname host.containers.internal reaches the podman host where llama.cpp listens on 7770.

The :7770 endpoint is the frank-default-large service, managed by the sibling vllmScripts repository (not this repo). Model swaps, context sizing, and spec tuning happen there. See the vllmScripts knowledge base.

Alternative backend: frank-default-small on port 52625 (FastFlowLM, AMD NPU, qwen3.6-moe:35b-a3b). Same OpenAI-compatible API shape. Point OPENAI_API_BASE_URL at http://host.containers.internal:52625/v1.

Volumes

Container path	Host path (convention)
`/app/backend/data`	`/opt/openwebui/data`

Healthcheck

OpenWebUI responds on /health with 200.

Gotchas

Use the non-CUDA image on AMD hosts.
The default admin account is created on first login; secure it immediately.
If the backend (llama.cpp) is slow to load, OpenWebUI may show connection errors until the model is ready — that matches the known 3–8 minute model load time.

2.4 KiB Raw Blame History Unescape Escape