dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Move frank-default-large from :52625 to :7770 (dankclaw)

Browse Source 
Operator request (2026-08-22): the primary inference server
(frank-default-large, the 27B CIRU model on Dankclaw) moves to the
standard llama.cpp port 7770. Both LLM services now listen on :7770,
distinguished by host (dankserver vs dankclaw).

Functional changes:
- scripts/frank-default-large.env: LISTEN_PORT 52625 -> 7770
- scripts/serve-frank-large.sh: default LISTEN_PORT 52625 -> 7770
- scripts/deploy.sh: LLM_HEALTH[dankclaw] -> :7770
- open-webui quadlet: OPENAI_API_BASE_URL -> 192.168.0.143:7770
  (re-pointed so chat keeps working; data dir preserved)
- scripts/bench-fast-greedy.sh: PORT 52625 -> 7770

Tests updated to the new topology (both LLM endpoints on :7770 by host;
open-webui -> :7770). Docs + pi KB updated; dated historical runbooks
keep their point-in-time ports. models.json (pi client) re-pointed to
:7770 separately, effective on the next client restart.
This commit is contained in:
Frank
2026-08-22 22:11:53 -07:00
parent e367f0f43c
commit fbfbc5c6db
15 changed files with 51 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Dankclaw/Dankclaw Stack.md
+3 -3
View File
@@ -1,7 +1,7 @@
# Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, and development stack. It
hosts the LLM inference endpoint (`frank-default-large`, `:52625`, CIRU
hosts the LLM inference endpoint (`frank-default-large`, `:7770`, CIRU
ROCmFPX llama.cpp), the agent tools, the privacy search, and the
observability stack. The llama.cpp endpoint (`frank-default-small`,
`:7770`) runs on Dankserver (192.168.0.235).
@@ -17,7 +17,7 @@ graph LR
S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770]
O --> H[Hermes]
O --> E[CIRU llama.cpp :52625]
O --> E[CIRU llama.cpp :7770]
```
## Services
@@ -32,7 +32,7 @@ graph LR
| Grafana | grafana.dankclaw.local | 3002 | [[Grafana]] |
| Prometheus | prometheus.dankclaw.local | 9090 | [[Prometheus]] |
| Homepage | dankclaw.local | 80/443 | — |
| frank-default-large (host service, Dankclaw) | — | 52625 | [[deployment]] |
| frank-default-large (host service, Dankclaw) | — | 7770 | [[deployment]] |
| frank-default-small (host service, Dankserver) | — | 7770 | [[deployment]] |
The last two rows are bare-metal systemd services, not containers. This
docs/Dankclaw/OpenWebUI.md
+5 -5
View File
@@ -11,19 +11,19 @@ OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compat
| Image | `ghcr.io/open-webui/open-webui:main` (CPU build; Dankclaw is CPU-only) |
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
| Default backend | vLLM on Dankclaw `:52625` = `frank-default-large` (via host LAN IP `192.168.0.143:52625`) |
| Default backend | CIRU llama.cpp on Dankclaw `:7770` = `frank-default-large` (via host LAN IP `192.168.0.143:7770`) |
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
## The LLM backends
The fleet runs two OpenAI-compatible endpoints. This mapping is the **live** state (verified 2026-08-19); it is the reverse of the 2026-08-17 note, which is now out of date.
The fleet runs two OpenAI-compatible endpoints. This mapping is the **live** state (verified 2026-08-22). Both services listen on `:7770`, distinguished by host. On 2026-08-22 `frank-default-large` moved from `:52625` to `:7770`.
| Endpoint | Model name | Host | Capability |
|----------|-----------|------|-----------|
| `:52625` (vLLM) | `frank-default-large` | Dankclaw | completion |
| `:7770` (llama.cpp FastMTP) | `frank-default-small` | Dankserver | completion, multimodal |
| `:7770` (CIRU llama.cpp) | `frank-default-large` | Dankclaw | completion |
| `:7770` (llama.cpp) | `frank-default-small` | Dankserver | completion, multimodal |
The default connection points at the local vLLM (`frank-default-large`), so chat stays on-host. To add the other backend, open **Admin → Connections** in the UI and point an OpenAI connection at `http://192.168.0.235:7770/v1` (key `none`).
@@ -61,7 +61,7 @@ The quadlet carries a built-in healthcheck (30s interval, 3 retries, 300s start
- Use the non-CUDA image on this CPU-only host.
- The default admin account is created on first login; secure it immediately.
- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
- The LLM is reached via the host LAN IP `192.168.0.143:52625`, not the bridge gateway: the gateway (`10.89.3.1`) lives in the rootless netns, while llama-server runs in the host netns, so the gateway refuses. `host.containers.internal` is slirp4netns-only and does not resolve on a bridge either. To reach the Dankserver backend, use the LAN address `192.168.0.235:7770`.
- The LLM is reached via the host LAN IP `192.168.0.143:7770`, not the bridge gateway: the gateway (`10.89.3.1`) lives in the rootless netns, while llama-server runs in the host netns, so the gateway refuses. `host.containers.internal` is slirp4netns-only and does not resolve on a bridge either. To reach the Dankserver backend, use the LAN address `192.168.0.235:7770`.
## History