From c353725ea51313bfd5bea52d54a7d35f14ca1d79 Mon Sep 17 00:00:00 2001 From: Frank Date: Sun, 16 Aug 2026 23:19:54 -0700 Subject: [PATCH] docs: cross-link the vllmScripts project (model endpoints + git host) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README: fix broken 'Also See' placeholder URL -> live Gitea LAN URL - index.md: expand vllmScripts See Also with the endpoints it serves - OpenWebUI.md: :7770 backend is the llama.cpp PP head (split to the 3090), managed by vllmScripts; note the FLM NPU :52625 alternative; fix the broken [[llama.cpp Server]] wikilink - Gitea.md: hosted repositories table (both projects) - Dankclaw Stack.md: host-level LLM endpoints in the service inventory - AGENTS.md: runtime note — LLM endpoints live in vllmScripts, not here Pre-existing failures (untracked ComfyUI WIP): tests 6, 8, 9, 10. Docs seams green: ok 19 (compose service docs), ok 20 (index links). --- AGENTS.md | 1 + README.md | 5 ++++- docs/Dankclaw/Dankclaw Stack.md | 11 +++++++++-- docs/Dankclaw/Gitea.md | 9 +++++++++ docs/Dankclaw/OpenWebUI.md | 13 +++++++++++-- docs/index.md | 5 ++++- 6 files changed, 38 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 27e8d52..73dd3e9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,6 +46,7 @@ Test seams: compose validity, env consistency (no leaked secrets), Caddyfile val ## Runtime & Troubleshooting +- The LLM endpoints are bare-metal systemd services managed by the sibling `vllmScripts` repository (`../vllmScripts`), not by this repo: llama.cpp PP head `:7770` and FastFlowLM NPU `:52625` on Dankclaw, RPC worker `:50052` on Dankserver. Do not edit their scripts here. - Model load on the LLM servers takes 3–8 minutes; `503 Loading model` is normal during load. Do not kill the process. - `sudo` on both boxes needs a tty. Do not use `echo password | sudo -S` in committed scripts; run sudo interactively. - Never run `pkill -f` with a pattern that appears in your own command line. Use `pkill -x `. diff --git a/README.md b/README.md index a35327c..f927b22 100644 --- a/README.md +++ b/README.md @@ -47,4 +47,7 @@ Run `./tests/run.sh` before every push. See [docs/Architecture/Testing Policy.md ## Also See -- [vllmScripts](https://gitea.dankclaw.local/.../vllmScripts) — the deployment model this repo follows +- [vllmScripts](http://dankclaw:3000/dank/vllmScripts) (local: `../vllmScripts`) — + the model-serving scripts this stack consumes: `frank-default-large` + (llama.cpp PP head, `:7770`, backend of OpenWebUI) and + `frank-default-small` (FastFlowLM NPU, `:52625`) diff --git a/docs/Dankclaw/Dankclaw Stack.md b/docs/Dankclaw/Dankclaw Stack.md index b4ea6b9..7d0cdcc 100644 --- a/docs/Dankclaw/Dankclaw Stack.md +++ b/docs/Dankclaw/Dankclaw Stack.md @@ -1,6 +1,6 @@ # Dankclaw Stack -Dankclaw (192.168.0.143) runs the AI, search, and development stack. It hosts the inference endpoint (llama.cpp, port 7770, from the `vllmScripts` repo), the agent tools, the privacy search, and the observability stack. +Dankclaw (192.168.0.143) runs the AI, search, and development stack. It hosts the inference endpoints (llama.cpp PP head `:7770`, FastFlowLM NPU `:52625`, both from the `vllmScripts` repo), the agent tools, the privacy search, and the observability stack. ```mermaid graph LR @@ -11,8 +11,9 @@ graph LR C --> F[ComfyUI] C --> D[Grafana] S --> V[Gluetun VPN] - O --> L[llama.cpp :7770] + O --> L[llama.cpp PP head :7770] O --> H[Hermes] + N[FastFlowLM NPU :52625] ``` ## Services @@ -27,6 +28,11 @@ graph LR | Grafana | grafana.dankclaw.local | 3002 | [[Grafana]] | | Prometheus | prometheus.dankclaw.local | 9090 | [[Prometheus]] | | Homepage | dankclaw.local | 80/443 | — | +| frank-default-large (host service) | — | 7770 | [vllmScripts](../../vllmScripts/docs/index.md) | +| frank-default-small (host service) | — | 52625 | [vllmScripts](../../vllmScripts/docs/index.md) | + +The last two rows are bare-metal systemd services, not containers. The +`vllmScripts` repository owns their scripts and knowledge base. ## Routing Rules @@ -39,3 +45,4 @@ graph LR - [[index]] — repository index - [[Network Architecture]] — routing and proxy - [[Observability]] — Grafana, Prometheus, logs +- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the model-serving endpoints on this host diff --git a/docs/Dankclaw/Gitea.md b/docs/Dankclaw/Gitea.md index fb398fd..91b1615 100644 --- a/docs/Dankclaw/Gitea.md +++ b/docs/Dankclaw/Gitea.md @@ -14,6 +14,15 @@ Gitea is a self-hosted Git service with issues, pull requests, and a web UI. On The containers share the private bridge network `systemd-gitea`. The quadlet network file `gitea.network` enables DNS, so the containers resolve each other by name. The automatic `podman` default network has DNS disabled. Do not run the Gitea containers on it. +## Hosted repositories + +| Repository | Purpose | +|------------|---------| +| `dank/serverAppConfigs` | this repo — Podman stack configs | +| `dank/vllmScripts` | model-serving scripts: llama.cpp PP head (`:7770`), FastFlowLM NPU (`:52625`), RPC worker (`:50052`) | + +The vllmScripts knowledge base: [../vllmScripts/docs/index.md](../../vllmScripts/docs/index.md) + ## Management Run all commands as `dank` on Dankclaw. diff --git a/docs/Dankclaw/OpenWebUI.md b/docs/Dankclaw/OpenWebUI.md index 9f0c62c..3576d8d 100644 --- a/docs/Dankclaw/OpenWebUI.md +++ b/docs/Dankclaw/OpenWebUI.md @@ -11,7 +11,7 @@ OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compat | Image | `ghcr.io/open-webui/open-webui:main` (CPU build for non-NVIDIA hosts) | | Port | 3000 (host) → 8080 (container) | | Data volume | config, sessions, RAG indexes | -| Backend | OpenAI-compatible endpoint at `http://localhost:7770/v1` (llama.cpp) | +| Backend | OpenAI-compatible endpoint at `http://localhost:7770/v1` — the llama.cpp PP head (Qwen3.8-27B, tensors split to the Dankserver RTX 3090 worker) | > Note: the `:cuda` image tag was previously used on Dankserver. Dankclaw is AMD; use the default (CPU) image, NOT the cuda tag. @@ -27,6 +27,15 @@ environment: The hostname `host.containers.internal` reaches the podman host where llama.cpp listens on 7770. +> The `:7770` endpoint is the `frank-default-large` service, managed by +> the sibling `vllmScripts` repository (not this repo). Model swaps, +> context sizing, and spec tuning happen there. See the +> [vllmScripts knowledge base](../../vllmScripts/docs/index.md). +> +> Alternative backend: `frank-default-small` on port 52625 (FastFlowLM, +> AMD NPU, `qwen3.6-moe:35b-a3b`). Same OpenAI-compatible API shape. Point +> `OPENAI_API_BASE_URL` at `http://host.containers.internal:52625/v1`. + ## Volumes | Container path | Host path (convention) | @@ -47,4 +56,4 @@ OpenWebUI responds on `/health` with 200. - [[Dankclaw Stack]] - [[Hermes]] — the agent OpenWebUI can talk to -- Knowledge base: [[llama.cpp Server]] +- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the llama.cpp PP head and FastFlowLM NPU endpoints diff --git a/docs/index.md b/docs/index.md index 091eb65..646a356 100644 --- a/docs/index.md +++ b/docs/index.md @@ -61,5 +61,8 @@ This repository tracks the container stacks for both hosts. Git is the single so ## See Also -- [vllmScripts repository](../vllmScripts/docs/index.md) — the deployment model this repo follows +- [vllmScripts repository](../vllmScripts/docs/index.md) — the model-serving + scripts on the same hosts: `frank-default-large` (llama.cpp PP head, + port 7770, consumed by OpenWebUI) and `frank-default-small` (FastFlowLM + NPU, port 52625). Git workflow model for this repo. - [[AGENTS.md]] — repository rules