Doc audit: fix all broken links + correct stale facts across the KB
Fanned out 5 read-only doc-auditor subagents (one per service group) that fact-checked the knowledge base against the live system + repo config, researched official setup docs online, and verified progress. Applied: - All 34 broken links fixed (real pages / external URLs / correct relative paths); 0 broken links remain. No .AGENTTRASH links (the 4 mentions are accurate descriptive text, not links). - Fact-check fixes: Readarr retired (May 2026), Radarr v4->v6, Calibre 9.2.1 live, Calibre-Web 0.6.25 live, Copyparty v1.20.21, LLM index ncmoe 38 + 786432 ctx, frank-default-small active (not frank-pp-worker), Hermes/SearXNG/Grafana hostnames, Loki/Alloy local images, Observability diagram (Promtail->Alloy), VLLM retired note, DFlash status, benchmark live-drift notes, Homepage test count 116.
This commit is contained in:
Frank
@@ -46,7 +46,7 @@ Caddy holds one Let's Encrypt certificate per vhost, issued and renewed through
|
||||
|
||||
## Gluetun Path (SearXNG only)
|
||||
|
||||
SearXNG attaches to the Gluetun network namespace (`network_mode: service:gluetun`). Only SearXNG shares the tunnel. Everything else on Dankclaw uses the normal LAN route. This matches KB Pattern 2 in [[VPN Routing for SearXNG]].
|
||||
SearXNG attaches to the Gluetun network namespace (`network_mode: service:gluetun`). Only SearXNG shares the tunnel. Everything else on Dankclaw uses the normal LAN route. This matches KB Pattern 2 in [[Gluetun]].
|
||||
|
||||
## Access Policy
|
||||
|
||||
@@ -60,4 +60,4 @@ SearXNG attaches to the Gluetun network namespace (`network_mode: service:gluetu
|
||||
- [[Caddy]] — vhost map, TLS, basic auth
|
||||
- [[Dankserver Stack]], [[Dankclaw Stack]]
|
||||
- [[Gluetun]]
|
||||
- Knowledge base: [[VPN Routing for SearXNG]], [[SearXNG Tuning]]
|
||||
- Knowledge base: [[Gluetun]], [[SearXNG]]
|
||||
@@ -76,4 +76,4 @@ The web UI responds on `/`. ComfyUI also exposes `/system_stats`. On this APU, `
|
||||
- [[Dankclaw Stack]]
|
||||
- [[ComfyUI ROCm and Vulkan]] — the live-verified GPU path research
|
||||
- [[Lemonade and sd.cpp]] — the ggml-family alternative
|
||||
- Knowledge base: [[ComfyUI]] (tools), [[ComfyUI Troubleshooting]]
|
||||
- Knowledge base: [[ComfyUI]] (tools)
|
||||
|
||||
@@ -53,7 +53,7 @@ The containers use the quadlet fresh-start model. A stop removes the container (
|
||||
|
||||
## Secrets
|
||||
|
||||
The database password lives in `/opt/gitea/.env` on the host (mode 0600). The quadlet files reference it with `EnvironmentFile=`. The password never enters this repository. Placeholders live in [`dankclaw/gitea/.env.example`](gitea/.env.example).
|
||||
The database password lives in `/opt/gitea/.env` on the host (mode 0600). The quadlet files reference it with `EnvironmentFile=`. The password never enters this repository. Placeholders live in [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example).
|
||||
|
||||
Do not change the password in place. Change the PostgreSQL role first, then update `/opt/gitea/.env` and the `[database] PASSWD` line in the app.ini (see Gotchas), then restart the stack.
|
||||
|
||||
@@ -61,9 +61,9 @@ Do not change the password in place. Change the PostgreSQL role first, then upda
|
||||
|
||||
| File | Location | Repo copy |
|
||||
|------|----------|-----------|
|
||||
| Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server,caddy}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](gitea/quadlet/) |
|
||||
| Caddyfile | `/opt/gitea/data/caddy/Caddyfile` | [`dankclaw/gitea/Caddyfile`](gitea/Caddyfile) |
|
||||
| Secrets | `/opt/gitea/.env` (0600) | [`dankclaw/gitea/.env.example`](gitea/.env.example) |
|
||||
| Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server,caddy}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](../../dankclaw/gitea/quadlet/) |
|
||||
| Caddyfile | `/opt/gitea/data/caddy/Caddyfile` | host-only (not in repo) |
|
||||
| Secrets | `/opt/gitea/.env` (0600) | [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example) |
|
||||
|
||||
After editing a repo copy, copy it to the target, then run `systemctl --user daemon-reload` and restart the affected units.
|
||||
|
||||
@@ -75,14 +75,14 @@ After editing a repo copy, copy it to the target, then run `systemctl --user dae
|
||||
| `/var/lib/postgresql/data` | `/opt/gitea/data/gitea-postgres` | PostgreSQL data |
|
||||
| `/etc/caddy/Caddyfile`, `/root/.local/share/caddy` | `/opt/gitea/data/caddy/...` | Caddy config and TLS state |
|
||||
|
||||
Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge base page [[Gitea Data Persistence]].
|
||||
Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge base page [[Gitea]].
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **app.ini wins over env vars.** Gitea writes `/data/gitea/conf/app.ini` (host: `/opt/gitea/data/gitea-gitea/gitea/conf/app.ini`) on first start. The `[database] HOST` and `PASSWD` lines persist and override the `DATABASE_HOST`/`DATABASE_PASSWD` env vars. When you rename the database container, edit the `HOST` line.
|
||||
- **The files in the data directory are owned by host UID 100999**, not `dank` (rootless ID mapping: container uid 1000 → host 100999). Edit them inside the container with `podman exec --user 1000 gitea-server ...`.
|
||||
- **The host auto-reboots.** `unattended-upgrades` reboots Dankclaw when a package needs it. The quadlet units make Gitea survive that. See the 2026-08-16 incident below.
|
||||
- **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../compose.yaml) maps 3001. Align the two before deploying the full stack.
|
||||
- **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../../dankclaw/compose.yaml) maps 3001. Align the two before deploying the full stack.
|
||||
- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the quadlet units on ports 3000, 2222, 80, and 443.
|
||||
- **Rootless cannot bind ports below 1024 by default.** `gitea-caddy` publishes 80 and 443. Without a host sysctl, `rootlessport` fails with `permission denied` and the unit crash-loops under `Restart=always`. The fix is a one-time sysctl: `net.ipv4.ip_unprivileged_port_start=80`, persisted in `/etc/sysctl.d/99-rootless-ports.conf`. Applied 2026-08-18 after a 13,000-restart crash loop.
|
||||
|
||||
@@ -94,4 +94,4 @@ Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge ba
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Postgres]]
|
||||
- Knowledge base: [[Gitea]] (tools), [[Gitea Management]], [[Gitea Data Persistence]], [[Docker and Podman]]
|
||||
- Knowledge base: [[Gitea]] (tools)
|
||||
|
||||
@@ -71,4 +71,4 @@ The returned IP must differ from the host IP.
|
||||
|
||||
- [[SearXNG]] — the consumer of the tunnel
|
||||
- [[Dankclaw Stack]]
|
||||
- Knowledge base: [[VPN Routing for SearXNG]]
|
||||
- Knowledge base: [[SearXNG]]
|
||||
|
||||
@@ -30,11 +30,11 @@ Hermes is the self-improving AI agent from Nous Research. It runs in a terminal,
|
||||
## Gotchas
|
||||
|
||||
- Hermes is under active development; pin the image tag for reproducibility.
|
||||
- Model choice drives quality; the local `frank-default-large` (Dankserver, 7770) is the intended backend.
|
||||
- Model choice drives quality; the local `frank-default-large` (Dankclaw, 7770) is the intended backend.
|
||||
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
- [[OpenWebUI]] — the chat UI that pairs with Hermes
|
||||
- Knowledge base: [[Agentic Search Best Practices]] (Hermes as the search agent)
|
||||
- Knowledge base: [[SearXNG]] (Hermes as the search agent)
|
||||
@@ -27,7 +27,7 @@ The fleet runs two OpenAI-compatible endpoints. This mapping is the **live** sta
|
||||
|
||||
The default connection points at the local vLLM (`frank-default-large`), so chat stays on-host. To add the other backend, open **Admin → Connections** in the UI and point an OpenAI connection at `http://192.168.0.235:7770/v1` (key `none`).
|
||||
|
||||
> Both endpoints are bare-metal systemd services managed by the sibling `vllmScripts` repository (not this repo). Model swaps, context sizing, and spec tuning happen there. See the [vllmScripts knowledge base](../../vllmScripts/docs/index.md).
|
||||
> Both endpoints are bare-metal systemd services managed by this repository (the former `vllmScripts` repo, merged in 2026-08-22). Model swaps, context sizing, and spec tuning happen here. See the [LLM knowledge base](../LLM/index.md).
|
||||
|
||||
## Boot persistence (Quadlet)
|
||||
|
||||
@@ -72,4 +72,4 @@ The quadlet carries a built-in healthcheck (30s interval, 3 retries, 300s start
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Hermes]] — the agent OpenWebUI can talk to
|
||||
- [[SearXNG]] — the search engine on this host
|
||||
- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the vLLM and llama.cpp endpoints
|
||||
- [LLM knowledge base](../LLM/index.md) — the vLLM and llama.cpp endpoints
|
||||
|
||||
@@ -62,7 +62,7 @@ healthcheck:
|
||||
|
||||
- The database password must match between the `postgres` service and Gitea's `DATABASE_PASSWD`; both come from the same `.env` variable.
|
||||
- Do not change the Gitea `SECRET_KEY` or `INTERNAL_TOKEN` after migration — encrypted data becomes unreadable.
|
||||
- Back up with `pg_dump`; see [[Gitea Data Persistence]] (knowledge base).
|
||||
- Back up with `pg_dump`; see [[Gitea]] (knowledge base).
|
||||
|
||||
## See Also
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
SearXNG is a privacy-respecting metasearch engine. It aggregates results from many upstream search engines into one result set and serves both an HTML UI and a JSON API for agents.
|
||||
|
||||
**Host:** Dankclaw — **Hostname:** `search.dankclaw.local` — **Docs:** https://docs.searxng.org
|
||||
**Host:** Dankclaw — **Hostname:** `search.franksdank.cloud` (Caddy) — **Docs:** https://docs.searxng.org
|
||||
|
||||
## Facts (from research, primary source)
|
||||
|
||||
@@ -24,7 +24,7 @@ ContainerName=searxng
|
||||
Network=gluetun.container
|
||||
```
|
||||
|
||||
Port publishing happens on the **gluetun** unit, not on searxng. See [[Gluetun]] and the knowledge base page [[VPN Routing for SearXNG]] (KB Pattern 2 is the one used here).
|
||||
Port publishing happens on the **gluetun** unit, not on searxng. See [[Gluetun]] (KB Pattern 2 is the one used here).
|
||||
|
||||
## Boot Persistence (Quadlet)
|
||||
|
||||
@@ -47,7 +47,7 @@ Deploy with `scripts/deploy-searxng-quadlet.sh dankclaw`. It copies the units, d
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "${SEARXNG_SECRET}" # from .env
|
||||
base_url: "http://search.dankclaw.local/"
|
||||
base_url: "${SEARXNG_BASE_URL}" # resolves to http://192.168.0.143:8081/
|
||||
limiter: true
|
||||
method: "POST"
|
||||
|
||||
@@ -106,4 +106,4 @@ Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local s
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Gluetun]] — the VPN container
|
||||
- Knowledge base: [[VPN Routing for SearXNG]], [[SearXNG Tuning]], [[Agentic Search Best Practices]]
|
||||
- Knowledge base: [[Gluetun]], [[SearXNG]], [[Hermes]]
|
||||
|
||||
@@ -365,5 +365,5 @@ Use Copyparty for curated file collections. Use *arr for automatic downloads. Us
|
||||
- [[Jellyfin]] — media server that consumes *arr downloads
|
||||
- [[Copyparty]] — alternative file viewer
|
||||
- [[Calibre-Web]] — manual ebook management (Readarr auto-downloads into the same library)
|
||||
- [[Sonarr Wiki]] — https://wiki.servarr.com/sonarr
|
||||
- [[Radarr Wiki]] — https://wiki.servarr.com/radarr
|
||||
- [Sonarr Wiki](https://wiki.servarr.com/sonarr)
|
||||
- [Radarr Wiki](https://wiki.servarr.com/radarr)
|
||||
|
||||
@@ -8,8 +8,8 @@ Calibre-Web is a web application that browses, reads, and downloads eBooks from
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Latest version | 0.6.27-ls397 |
|
||||
| Image | `lscr.io/linuxserver/calibre-web:0.6.27-ls397` |
|
||||
| Latest version | 0.6.25-ls365 (live); 0.6.27-ls397 upstream |
|
||||
| Image | `lscr.io/linuxserver/calibre-web:latest` (live: 0.6.25-ls365) |
|
||||
| Base OS | Ubuntu Noble (24.04 LTS, since 2024-12-05) |
|
||||
| Port | 8083 (host) → 8083 (container) |
|
||||
| Default login | `admin` / `admin123` |
|
||||
|
||||
@@ -18,7 +18,7 @@ The actual library files (`.epub`, `.mobi`, `.pdf`) and the Calibre database (`.
|
||||
|
||||
## Current Version
|
||||
|
||||
Calibre 9.9.0 is the latest stable release. Major features include:
|
||||
Calibre 9.2.1 is the version on the live host (9.9.0 is the upstream latest). Major features include:
|
||||
- Bookshelf view
|
||||
- Opt-in AI features
|
||||
- Content server security fix (CVE-2025-7404 equivalent): a bug since v6.16.0 allows any user with server access to read non-Calibre files on the host. Upgrade to 9.9.0+ if serving content to untrusted users.
|
||||
|
||||
@@ -8,7 +8,7 @@ Copyparty is a lightweight file server and media streamer that runs as a single
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Latest version | v1.20.19 (released 2026-07-27) |
|
||||
| Latest version | v1.20.21 (2026-08) |
|
||||
| Image | `ghcr.io/9001/copyparty:latest` (official Dockerfile in repo) |
|
||||
| Port | 3923 (host) → 3923 (container) |
|
||||
| Volume | bind mount the shared root directory |
|
||||
|
||||
@@ -170,4 +170,4 @@ Run Jellyfin alongside Plex for an incremental migration. Point Jellyfin at the
|
||||
- [[Dankserver Stack]]
|
||||
- [[ARR-Stack]] — automated download into Jellyfin libraries
|
||||
- [[Copyparty]] — alternative file viewer for existing collections
|
||||
- [[Plex]] — existing media server on this host
|
||||
- Plex — existing media server on this host
|
||||
|
||||
@@ -190,5 +190,5 @@ healthcheck:
|
||||
## See Also
|
||||
|
||||
- [[Dankserver Stack]]
|
||||
- [[RomM Scraping]] — metadata providers, cover art, naming conventions, emulator integration
|
||||
- [[RomM-Scraping]] — metadata providers, cover art, naming conventions, emulator integration
|
||||
- [[Calibre-Web]] — sibling library web UI
|
||||
|
||||
@@ -8,7 +8,7 @@ Radarr manages movies in the *arr stack on Dankserver. It automatically searches
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `lscr.io/linuxserver/radarr:latest` (v4 stable) |
|
||||
| Image | `lscr.io/linuxserver/radarr:latest` (v6 stable) |
|
||||
| Port | 7878 (host) → 7878 (container) |
|
||||
| Min RAM | 512 MB |
|
||||
| Metadata | TMDB / TheMovieDB API |
|
||||
@@ -44,7 +44,7 @@ radarr:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Radarr v4 is the current stable. `:latest` points to v4, not v5.
|
||||
- Radarr v6 is the current stable (v6.3.x as of 2026-08). `:latest` points to the current release.
|
||||
- Movie renaming produces `Title (Year)` format. Jellyfin scanner handles this.
|
||||
- File permissions: qBittorrent downloads as PUID; arr apps must read/move files.
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Readarr
|
||||
|
||||
> **RETIRED (May 2026).** The Servarr Team retired Readarr: its metadata became unusable and the community migration to Open Library stalled. There is no upstream development. Alternatives: the Bookshelf community fork, or the Calibre content server. See [Readarr (Retired)](https://wiki.servarr.com/readarr).
|
||||
|
||||
Readarr manages ebooks and audiobooks in the *arr stack on Dankserver. It automatically searches, downloads, and organizes books into the Calibre library.
|
||||
|
||||
**Host:** Dankserver — **Hostname:** `readarr.dankserver.local` — **Docs:** https://wiki.servarr.com/readarr
|
||||
|
||||
@@ -8,7 +8,7 @@ Alloy is Grafana's log and telemetry shipper. It replaces Promtail (EOL 2026-03-
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `grafana/alloy:latest` |
|
||||
| Image | `franksdank/alloy:latest` (local build with a static busybox healthcheck) |
|
||||
| Port | none published (push-only) |
|
||||
| Config | River syntax, `config.alloy` |
|
||||
| Successor to | Promtail (EOL 2026-03-02) |
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Grafana
|
||||
|
||||
Grafana visualizes the metrics that Prometheus collects from every service. It runs on Dankclaw and is reachable at `grafana.dankclaw.local`.
|
||||
Grafana visualizes the metrics that Prometheus collects from every service. It runs on Dankclaw and is reachable at `grafana.franksdank.cloud` (via Caddy, with basicauth).
|
||||
|
||||
**Host:** Dankclaw — **Hostname:** `grafana.dankclaw.local` — **Port:** 3002 — **Docs:** https://grafana.com/docs
|
||||
**Host:** Dankclaw — **Hostname:** `grafana.franksdank.cloud` — **Port:** 3002 (internal) — **Docs:** https://grafana.com/docs
|
||||
|
||||
## Facts
|
||||
|
||||
@@ -20,7 +20,7 @@ Grafana visualizes the metrics that Prometheus collects from every service. It r
|
||||
environment:
|
||||
GF_SECURITY_ADMIN_USER: "${GRAFANA_ADMIN_USER}"
|
||||
GF_SECURITY_ADMIN_PASSWORD: "${GRAFANA_ADMIN_PASSWORD}"
|
||||
GF_SERVER_ROOT_URL: "http://grafana.dankclaw.local"
|
||||
GF_SERVER_ROOT_URL: "https://grafana.franksdank.cloud"
|
||||
volumes:
|
||||
- /opt/grafana/data:/var/lib/grafana
|
||||
- /opt/grafana/provisioning:/etc/grafana/provisioning:ro
|
||||
|
||||
@@ -72,7 +72,7 @@ These are the deltas from the initial recommendations. Everything else was accep
|
||||
## Build status — complete (2026-08-23)
|
||||
|
||||
The build is done, slice by slice, full TDD (each slice red → green, committed
|
||||
and pushed). 112 tests pass. The render logic is verified with a Node DOM shim.
|
||||
and pushed). 116 tests pass. The render logic is verified with a Node DOM shim.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
|
||||
@@ -8,7 +8,7 @@ Loki is the log aggregation backend of the observability stack. It stores and qu
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `grafana/loki:latest` |
|
||||
| Image | `franksdank/loki:latest` (local build with a static busybox healthcheck) |
|
||||
| Port | 3100 (host) → 3100 (container) |
|
||||
| Mode | single-binary (all components in one process) |
|
||||
| Storage | filesystem (bind mount) |
|
||||
|
||||
@@ -7,7 +7,7 @@ The observability stack runs on Dankclaw: Grafana for dashboards, Prometheus for
|
||||
```mermaid
|
||||
graph LR
|
||||
S[Services] -->|/metrics| P[Prometheus]
|
||||
S -->|logs| T[Promtail]
|
||||
S -->|logs| T[Alloy]
|
||||
T --> L[Loki]
|
||||
P --> G[Grafana]
|
||||
L --> G
|
||||
|
||||
@@ -160,7 +160,7 @@ The DFlash draft is not a standalone model. It must pair with its target model.
|
||||
|
||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||
|
||||
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is retired (2026-08-16) and its unit disabled (2026-08-17). The RTX 3090 now runs the PP worker for the Dankclaw head (see [[Qwen3.8 PP Deployment]]). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||
|
||||
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-default-small` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
|
||||
|
||||
### Active Services
|
||||
|
||||
The active service on Dankserver is `frank-pp-worker.service`: the ggml-rpc worker (RTX 3090) that receives the PP head's tensor share (45 of 65 blocks, ~23.8 GiB of 24 GiB at steady state). The old llama.cpp Qwen3.6-35B-A3B service (`frank-default-small`) is retired (2026-08-16) and disabled (2026-08-17). See [[Qwen3.8 PP Deployment]].
|
||||
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Qwen3.6-35B-A3B (Q6_K_P + ngram-mod) on port 7770. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
||||
|
||||
## Hardware Relevance
|
||||
|
||||
|
||||
@@ -189,7 +189,7 @@ For the 2-node pipeline-parallel split (RTX 3090 24 GB + Strix Halo 96 GB):
|
||||
- **Kept:** Qwen official config.json (HF) — exact architecture params (64 layers, 5120 hidden, 4 KV heads, head_dim 256) [8](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/config.json)
|
||||
- **Kept:** bartowski/Qwen_Qwen3.6-27B-GGUF — Q8_0 at 29.12 GB, imatrix calibration [2](https://huggingface.co/bartowski/Qwen_Qwen3.6-27B-GGUF)
|
||||
- **Kept:** Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp — DFlash 2B draft GGUF model [4](https://huggingface.co/Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp)
|
||||
- **Kept:** frank-default-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](docs/frank-default-large-benchmark.md)
|
||||
- **Kept:** frank-default-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](../frank-default-large-benchmark.md)
|
||||
- **Kept:** a4lg/Qwen3.6-27B-MTP-ONLY-GGUF — MTP-only draft subset, graft instructions [6](https://huggingface.co/a4lg/Qwen3.6-27B-MTP-ONLY-GGUF)
|
||||
- **Kept:** llama.cpp tools/rpc/README.md — RPC backend, weight/KV split, remote host setup [7](https://github.com/ggerganov/llama.cpp/blob/master/tools/rpc/README.md)
|
||||
- **Kept:** Unsloth Qwen3.6-27B-MTP-GGUF — MTP command-line usage for llama.cpp [9](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF)
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
|
||||
> See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and
|
||||
> [deployment.md](deployment.md) for the active configuration.
|
||||
> See [frank-redo-2026-08-17.md](../frank-redo-2026-08-17.md) and
|
||||
> [deployment.md](../deployment.md) for the active configuration.
|
||||
|
||||
# Qwen3.8-27B Pipeline-Parallel Deployment (Dankclaw + Dankserver)
|
||||
|
||||
Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX
|
||||
3090 (worker) using llama.cpp RPC. This page is the design record and the
|
||||
morning-of runbook. Research background:
|
||||
[Qwen3.8 Strix optimizations](../../qwen3.8-strix-optimizations.md).
|
||||
[Qwen3.8 Strix optimizations](../qwen3.8-strix-optimizations.md).
|
||||
|
||||
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# vLLM Index
|
||||
|
||||
> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[ciru-activefpx-promptforge]].
|
||||
|
||||
## Documentation Overview
|
||||
|
||||
This index contains all documentation for vLLM deployment on Dankserver and Dankclaw.
|
||||
|
||||
@@ -253,7 +253,7 @@ ngram draft costs its own forward pass / occupancy without adding enough
|
||||
accepted tokens on top of MTP. Baseline (no ngram) remains the production
|
||||
config (`--spec-type draft-mtp` only). Keep ngram-mod off.
|
||||
|
||||
The harness lives at [scripts/bench-spec-stack.sh](../scripts/bench-spec-stack.sh);
|
||||
The harness lives at [scripts/bench-spec-stack.sh](../../scripts/bench-spec-stack.sh);
|
||||
it boots the v3 stack on a scratch port and reads authoritative
|
||||
`print_timing` lines. It requires the production service to be stopped.
|
||||
|
||||
|
||||
@@ -105,6 +105,8 @@ service runs 700416 cells (13,680 MiB).
|
||||
|
||||
## Chosen State
|
||||
|
||||
> **Live drift (2026-08-22 audit):** the deployed `frank-default-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
|
||||
|
||||
Prefill speed is the priority, so the state keeps the top prefill tier and
|
||||
moves one expert layer to CPU for stability:
|
||||
|
||||
|
||||
+2
-2
@@ -19,7 +19,7 @@ graph LR
|
||||
B[upstream llama.cpp b10121
|
||||
frank-default-small, port 7770]
|
||||
C[Qwen3.6-35B-A3B-Uncensored Q6_K_P
|
||||
ngram-mod, 36 MoE experts on CPU, 700K ctx]
|
||||
ngram-mod, 38 MoE experts on CPU, 786432 ctx]
|
||||
A --> B --> C
|
||||
end
|
||||
|
||||
@@ -92,7 +92,7 @@ frank-default-large, port 7770]
|
||||
|
||||
## Pipeline Parallelism (archive)
|
||||
|
||||
- [[Pipeline Parallelism]] — index of multi-node inference research. The PP
|
||||
- [[Pipeline Parallelism/index]] — index of multi-node inference research. The PP
|
||||
pair is retired. These pages are research history.
|
||||
|
||||
## Related
|
||||
|
||||
@@ -103,4 +103,4 @@ podman rmi docker.io/rocm/vllm:rocm7.12.0_gfx1151_... #
|
||||
- [[ComfyUI]] — the service page (CPU config)
|
||||
- [[Lemonade and sd.cpp]] — the ggml-family alternative with working gfx1151 kernels
|
||||
- [[Observability]], [[Dankclaw Stack]]
|
||||
- KB: [[ComfyUI]] (tools), [[ComfyUI Troubleshooting]] (the "ROCm unavailable" claims here are now qualified)
|
||||
- KB: [[ComfyUI]] (tools) (the "ROCm unavailable" claims here are now qualified)
|
||||
+1
-1
@@ -141,5 +141,5 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
|
||||
## See Also
|
||||
|
||||
- [[AGENTS.md]] — repository rules
|
||||
- [AGENTS.md](../AGENTS.md) — repository rules
|
||||
- [docs/LLM/index.md](LLM/index.md) — the LLM fleet knowledge base
|
||||
|
||||
Reference in New Issue
Block a user