dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update sibling KB for the 2026-08-17 fleet redo (llama.cpp moved to Dankserver, vLLM on Dankclaw; OpenWebUI/Hermes backend addresses)

Browse Source
This commit is contained in:
Frank committed 2026-08-17 18:10:57 -07:00
1 parent c81933ccb4
commit 1de43f6777
7 files changed
+153 -21
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/Dankclaw/OpenWebUI.md
+12 -11
View File
@@ -11,30 +11,31 @@ OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compat
| Image | `ghcr.io/open-webui/open-webui:main` (CPU build for non-NVIDIA hosts) |
| Port | 3000 (host) → 8080 (container) |
| Data volume | config, sessions, RAG indexes |
| Backend | OpenAI-compatible endpoint at `http://localhost:7770/v1` — the llama.cpp PP head (Qwen3.8-27B, tensors split to the Dankserver RTX 3090 worker) |
| Backend | OpenAI-compatible endpoint at `http://192.168.0.235:7770/v1` — llama.cpp FastMTP on Dankserver (`frank-default-large`, Qwen3.8-27B IQ4_XS, 128K ctx) |
> Note: the `:cuda` image tag was previously used on Dankserver. Dankclaw is AMD; use the default (CPU) image, NOT the cuda tag.
## Connecting to llama.cpp
OpenWebUI talks to the local llama.cpp server through its OpenAI-compatible API. Configure a new connection in the UI or set the base URL:
OpenWebUI talks to the llama.cpp server on Dankserver through its OpenAI-compatible API. Configure a new connection in the UI or set the base URL:
```yaml
environment:
OPENAI_API_BASE_URL: "http://host.containers.internal:7770/v1"
OPENAI_API_BASE_URL: "http://192.168.0.235:7770/v1"
OPENAI_API_KEY: "none"
```
The hostname `host.containers.internal` reaches the podman host where llama.cpp listens on 7770.
The endpoint moved from Dankclaw to Dankserver in the 2026-08-17 redo; use the LAN address, not `host.containers.internal` (which only reaches the podman host).
> The `:7770` endpoint is the `frank-default-large` service, managed by
> the sibling `vllmScripts` repository (not this repo). Model swaps,
> context sizing, and spec tuning happen there. See the
> The `:7770` endpoint is the `frank-default-large` service on Dankserver,
> managed by the sibling `vllmScripts` repository (not this repo). Model
> swaps, context sizing, and spec tuning happen there. See the
> [vllmScripts knowledge base](../../vllmScripts/docs/index.md).
>
> Alternative backend: `frank-default-small` on port 52625 (FastFlowLM,
> AMD NPU, `qwen3.6-moe:35b-a3b`). Same OpenAI-compatible API shape. Point
> `OPENAI_API_BASE_URL` at `http://host.containers.internal:52625/v1`.
> Alternative backend: `frank-default-small` on port 52625 (vLLM on
> Dankclaw, nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8, 262K ctx,
> tool calling). Same OpenAI-compatible API shape. Point
> `OPENAI_API_BASE_URL` at `http://192.168.0.143:52625/v1`.
## Volumes
@@ -56,4 +57,4 @@ OpenWebUI responds on `/health` with 200.
- [[Dankclaw Stack]]
- [[Hermes]] — the agent OpenWebUI can talk to
- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the llama.cpp PP head and FastFlowLM NPU endpoints
- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the llama.cpp FastMTP and vLLM endpoints (Dankserver :7770, Dankclaw :52625)