docs: rename frank-default-large/small/sightsound to frank-large/small/npu across vault

Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
This commit is contained in:
Frank committed 2026-09-03 16:17:14 -07:00
1 parent 5a083eee6c
commit 8212d4451a
68 files changed
+1053 -1207

No files matched your search

+20 -20
View File
@@ -8,10 +8,10 @@ one checkout per host, one deploy script.
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty, - **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
Calibre-Web, the *arr stack, Jellyfin) as bare containers, plus the Calibre-Web, the *arr stack, Jellyfin) as bare containers, plus the
`frank-default-small` LLM service. `frank-small` LLM service.
- **Dankclaw** (192.168.0.143) — AI/search/development/observability stack - **Dankclaw** (192.168.0.143) — AI/search/development/observability stack
(SearXNG via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, (SearXNG via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus,
Loki, Alloy, Pi-hole) plus the `frank-default-large` LLM service. Caddy on Loki, Alloy, Pi-hole) plus the `frank-large` LLM service. Caddy on
Dankclaw reverse-proxies both hosts. Dankclaw reverse-proxies both hosts.
**2026-08-22 merge:** the former `vllmScripts` repository (LLM fleet scripts, **2026-08-22 merge:** the former `vllmScripts` repository (LLM fleet scripts,
@@ -26,9 +26,9 @@ host.
| Target | Runtime | Service | Model | Port | | Target | Runtime | Service | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | FreeToken v0.1.2 | frank-default-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 | | Dankserver | FreeToken v0.1.2 | frank-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
| Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-default-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 | | Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 |
| Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-default-sightsound | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 7771 | | Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-npu | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 7771 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -47,7 +47,7 @@ Qwen3.6-35B-A3B-FP8. On 2026-08-30 both services swapped again:
Dankclaw to the kingjones STRIX model on the card's pinned Dankclaw to the kingjones STRIX model on the card's pinned
kingjones30/ROCmFPX runtime, Dankserver to the Huihui abliterated kingjones30/ROCmFPX runtime, Dankserver to the Huihui abliterated
FP8 checkpoint on FreeToken. On 2026-08-30 the NPU endpoint returned as FP8 checkpoint on FreeToken. On 2026-08-30 the NPU endpoint returned as
`frank-default-sightsound` — FLM v1.0.1 on XDNA2 serving sight+sound (see the `frank-npu` — FLM v1.0.1 on XDNA2 serving sight+sound (see the
Runtime & Troubleshooting section and its runbook). Runbooks: Runtime & Troubleshooting section and its runbook). Runbooks:
[serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md). [serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`. Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
@@ -117,13 +117,13 @@ selected service and records its successful revision outside the checkout.
./scripts/deploy.sh dankclaw ./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankserver ./scripts/deploy.sh dankserver
./scripts/deploy.sh dankclaw --service searxng ./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-default-small ./scripts/deploy.sh dankserver --service frank-small
./scripts/deploy.sh dankclaw --dry-run ./scripts/deploy.sh dankclaw --dry-run
``` ```
A documentation-only change is a no-op. The LLM services are selected only by A documentation-only change is a no-op. The LLM services are selected only by
their own unit/env/serve files or the shared chat template, so unrelated their own unit/env/serve files or the shared chat template, so unrelated
container changes cannot interrupt inference. `frank-default-npu` is relink-only container changes cannot interrupt inference. `frank-npu` is relink-only
when explicitly selected. when explicitly selected.
Set `FRANK_SUDO_PW` for headless runs on a host without passwordless sudo Set `FRANK_SUDO_PW` for headless runs on a host without passwordless sudo
@@ -263,10 +263,10 @@ ALWAYS bench using llama-bench for llama.cpp or vllm bench for vllm runtimes.
DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE OFFICIAL BENCHES. ASK THE USER BEFORE DEVIATING FROM THIS DIRECTIVE. DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE OFFICIAL BENCHES. ASK THE USER BEFORE DEVIATING FROM THIS DIRECTIVE.
1. Stop the service before any benchmark: 1. Stop the service before any benchmark:
`sudo systemctl stop frank-default-large` (Dankclaw) or `sudo systemctl stop frank-large` (Dankclaw) or
`sudo systemctl stop frank-default-small` (Dankserver). `sudo systemctl stop frank-small` (Dankserver).
Then mask it to prevent the self-healing unit from restarting: Then mask it to prevent the self-healing unit from restarting:
`sudo systemctl mask frank-default-large` (Dankclaw). `sudo systemctl mask frank-large` (Dankclaw).
**CRITICAL WARNING:** Do NOT use `systemctl disable` on these service units. **CRITICAL WARNING:** Do NOT use `systemctl disable` on these service units.
On these hosts, `systemctl disable` doesn't just remove symlinks — it **deletes On these hosts, `systemctl disable` doesn't just remove symlinks — it **deletes
the unit file itself** from `/etc/systemd/system/`. You'd have to manually the unit file itself** from `/etc/systemd/system/`. You'd have to manually
@@ -279,7 +279,7 @@ DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE
(the model + KV fill the card), so changes are validated directly against (the model + KV fill the card), so changes are validated directly against
the service itself. the service itself.
3. Restart the service when the bench completes: unmask, then start: 3. Restart the service when the bench completes: unmask, then start:
`sudo systemctl unmask frank-default-large && sudo systemctl start frank-default-large` (Dankclaw). `sudo systemctl unmask frank-large && sudo systemctl start frank-large` (Dankclaw).
4. Verify health before reporting: 4. Verify health before reporting:
`curl http://192.168.0.235:7770/health` or `curl http://192.168.0.235:7770/health` or
`curl http://192.168.0.143:7770/health`. `curl http://192.168.0.143:7770/health`.
@@ -289,7 +289,7 @@ DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE
The Pi agent reaches the fleet through `/home/dank/.pi/agent/models.json`. The Pi agent reaches the fleet through `/home/dank/.pi/agent/models.json`.
This file lives outside the repository. Each provider is one host endpoint. This file lives outside the repository. Each provider is one host endpoint.
Each model `id` is the served alias. The two aliases are Each model `id` is the served alias. The two aliases are
`frank-default-small` (Dankserver `:7770`) and `frank-default-large` `frank-small` (Dankserver `:7770`) and `frank-large`
(Dankclaw `:7770`). (Dankclaw `:7770`).
The model `id` values are stable. The file The model `id` values are stable. The file
@@ -310,7 +310,7 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting ## Runtime & Troubleshooting
### frank-default-sightsound — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE) ### frank-npu — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE)
The third inference endpoint on Dankclaw. The `flm serve` runtime (dpkg The third inference endpoint on Dankclaw. The `flm serve` runtime (dpkg
package `fastflowlm`, v1.0.1, binary `/usr/bin/flm`) serves the **whole local package `fastflowlm`, v1.0.1, binary `/usr/bin/flm`) serves the **whole local
@@ -333,19 +333,19 @@ Hard requirements verified at first deploy (2026-08-30):
bind needs PASID/SVA, which requires enabled IOMMU hardware. Fix applied: bind needs PASID/SVA, which requires enabled IOMMU hardware. Fix applied:
`iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps the `iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps the
iGPU default domain identity-mapped — zero DMA-translation overhead for iGPU default domain identity-mapped — zero DMA-translation overhead for
frank-default-large — while enabling IOMMU + PASID; PASID domains are frank-large — while enabling IOMMU + PASID; PASID domains are
paged regardless of the default-domain type). Never set `iommu=off` on paged regardless of the default-domain type). Never set `iommu=off` on
this host again; it silently kills the NPU. this host again; it silently kills the NPU.
- Health probe = `GET /v1/version` (FLM has no `/health` route). - Health probe = `GET /v1/version` (FLM has no `/health` route).
- Unit: `User=dank`, `LimitMEMLOCK=infinity` (FLM requires unlimited - Unit: `User=dank`, `LimitMEMLOCK=infinity` (FLM requires unlimited
memlock), `Restart=on-failure`, `RestartSec=10`. Deploy via memlock), `Restart=on-failure`, `RestartSec=10`. Deploy via
`scripts/deploy.sh dankclaw --service frank-default-sightsound`. `scripts/deploy.sh dankclaw --service frank-npu`.
- Verified live at first deploy: image probe (red PNG as `image_url` - Verified live at first deploy: image probe (red PNG as `image_url`
data-URI → correct color answer, vision tower consumed the pixels) and an data-URI → correct color answer, vision tower consumed the pixels) and an
audio control-pair probe (tone WAV via `input_audio` consumed ~22 audio control-pair probe (tone WAV via `input_audio` consumed ~22
audio-tower tokens vs a text-only control). audio-tower tokens vs a text-only control).
Runbook: [serverAppConfigs-docs/LLM/frank-default-sightsound.md](serverAppConfigs-docs/LLM/frank-default-sightsound.md). Runbook: [serverAppConfigs-docs/LLM/frank-npu.md](serverAppConfigs-docs/LLM/frank-npu.md).
### kingjones Qwen3.8-Flash-Next STRIX on kingjones30/ROCmFPX (Dankclaw, ACTIVE) ### kingjones Qwen3.8-Flash-Next STRIX on kingjones30/ROCmFPX (Dankclaw, ACTIVE)
The large service serves The large service serves
@@ -375,7 +375,7 @@ correct here — mmap pages ARE page cache) and **never pin the PLE
tensor with `-ot`** (23 → 13.4 t/s). The 51.2B-param PLE table never tensor with `-ot`** (23 → 13.4 t/s). The 51.2B-param PLE table never
enters GPU memory (~20 bytes/token-file per 500 decoded). On-host file enters GPU memory (~20 bytes/token-file per 500 decoded). On-host file
sha256 verified == HF LFS oid `c6770d74...49fa`. Env sha256 verified == HF LFS oid `c6770d74...49fa`. Env
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`. `scripts/frank-large.env`, serve `scripts/serve-frank-large.sh`.
Runbook: Runbook:
[serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md](serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md). [serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md](serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md).
@@ -416,9 +416,9 @@ validated `--memory-num-gpu-blocks-experts 2048
(never grow the expert pool). 262144 context, reasoning on (the (never grow the expert pool). 262144 context, reasoning on (the
checkpoint ships the froggeric fixed chat template — no override). checkpoint ships the froggeric fixed chat template — no override).
FreeToken's API REQUIRES an explicit `"model"` field in chat requests FreeToken's API REQUIRES an explicit `"model"` field in chat requests
(422 without it). Env `scripts/frank-default-small.env`, serve (422 without it). Env `scripts/frank-small.env`, serve
`scripts/serve-frank-small.sh`. Runbook: `scripts/serve-frank-small.sh`. Runbook:
[serverAppConfigs-docs/LLM/freetoken-frank-default-small.md](serverAppConfigs-docs/LLM/freetoken-frank-default-small.md). [serverAppConfigs-docs/LLM/freetoken-frank-small.md](serverAppConfigs-docs/LLM/freetoken-frank-small.md).
**Serving status (2026-08-30):** swapped the model within the FreeToken **Serving status (2026-08-30):** swapped the model within the FreeToken
runtime (official FP8 → Huihui abliterated FP8). ~40 tok/s verified on runtime (official FP8 → Huihui abliterated FP8). ~40 tok/s verified on
+4 -4
View File
@@ -12,8 +12,8 @@ one deploy script.
| Host | User | IP | Podman stack | LLM service | LLM port | | Host | User | IP | Podman stack | LLM service | LLM port |
|------|------|----|--------------|-------------|----------| |------|------|----|--------------|-------------|----------|
| dankserver | dank | 192.168.0.235 | media and library | frank-default-small (llama.cpp, Qwen3.6-35B-A3B) | 7770 | | dankserver | dank | 192.168.0.235 | media and library | frank-small (llama.cpp, Qwen3.6-35B-A3B) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-default-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 | | dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
## Active Services ## Active Services
@@ -22,7 +22,7 @@ one deploy script.
| RomM | dankserver | romm.dankserver.local | 8085 | | RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 | | Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 | | Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-default-small | dankserver | — (bare endpoint) | 7770 | | frank-small | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 | | SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 | | ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 | | OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
@@ -30,7 +30,7 @@ one deploy script.
| Gitea | dankclaw | gitea.dankclaw.local | 3001 | | Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 | | Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 | | Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-default-large | dankclaw | — (bare endpoint) | 7770 | | frank-large | dankclaw | — (bare endpoint) | 7770 |
## Structure ## Structure
+15 -15
View File
@@ -10,20 +10,20 @@ model:
provider: dankclaw-large provider: dankclaw-large
base_url: http://192.168.0.143:7770/v1 base_url: http://192.168.0.143:7770/v1
api_key: not-needed api_key: not-needed
default: frank-default-large default: frank-large
providers: providers:
dankclaw-large: dankclaw-large:
api: http://192.168.0.143:7770/v1 api: http://192.168.0.143:7770/v1
discover_models: false discover_models: false
models: models:
frank-default-large: frank-large:
context_length: 262144 context_length: 262144
dankserver-small: dankserver-small:
api: http://192.168.0.235:7770/v1 api: http://192.168.0.235:7770/v1
discover_models: false discover_models: false
models: models:
frank-default-small: frank-small:
context_length: 163840 context_length: 163840
dankclaw-npu: dankclaw-npu:
api: http://192.168.0.143:7771/v1 api: http://192.168.0.143:7771/v1
@@ -36,18 +36,18 @@ providers:
# custom endpoint shape and are retained here as a settings backup. # custom endpoint shape and are retained here as a settings backup.
auxiliary: auxiliary:
vision: {provider: dankclaw-npu, model: gemma4-it:e4b, base_url: http://192.168.0.143:7771/v1} vision: {provider: dankclaw-npu, model: gemma4-it:e4b, base_url: http://192.168.0.143:7771/v1}
web_extract: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} web_extract: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
compression: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} compression: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
skills_hub: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} skills_hub: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
approval: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} approval: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
mcp: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} mcp: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
title_generation: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} title_generation: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
memory_query_rewrite: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} memory_query_rewrite: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
tts_audio_tags: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} tts_audio_tags: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
triage_specifier: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} triage_specifier: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
kanban_decomposer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} kanban_decomposer: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
profile_describer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} profile_describer: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
curator: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1} curator: {provider: custom, model: frank-small, base_url: http://192.168.0.235:7770/v1}
dashboard: dashboard:
# basic_auth intentionally omitted; its password_hash remains live-only. # basic_auth intentionally omitted; its password_hash remains live-only.
@@ -44,8 +44,9 @@ graph LR
```bash ```bash
./scripts/deploy.sh dankclaw ./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankserver ./scripts/deploy.sh dankserver
./scripts/deploy.sh frankbook-pro --service qbittorrent # podman-machine target
./scripts/deploy.sh dankclaw --service searxng ./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-default-small ./scripts/deploy.sh dankserver --service frank-small
./scripts/deploy.sh dankclaw --dry-run ./scripts/deploy.sh dankclaw --dry-run
``` ```
@@ -61,12 +62,12 @@ service on the other host.
|---|---|---| |---|---|---|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. | | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. | | SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
| `frank-default-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. | | `frank-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
| `frank-default-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. | | `frank-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. | | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
The shared chat template maps to the inference service on the selected target. The shared chat template maps to the inference service on the selected target.
The inactive `frank-default-npu` unit is relink-only when explicitly selected; The inactive `frank-npu` unit is relink-only when explicitly selected;
it is not restarted by an unrelated large-model deployment. it is not restarted by an unrelated large-model deployment.
## Caddy safety ## Caddy safety
+4 -1
View File
@@ -2,7 +2,10 @@
ComfyUI is a node-based AI image generation engine. Workflows are built from nodes that load models, sample latents, and post-process images; everything is scriptable through a web UI and an API. ComfyUI is a node-based AI image generation engine. Workflows are built from nodes that load models, sample latents, and post-process images; everything is scriptable through a web UI and an API.
**Host:** Dankclaw — **Hostname:** `comfyui.dankclaw.local` — **Port:** 8188 — **Backend:** ROCm GPU (gfx1151) — **Docs:** https://github.com/Comfy-Org/ComfyUI **Host:** Dankclaw — **Port:** `8188` published on the host; **no vhost and no
guard chain** — ComfyUI is LAN-only by construction (the Caddyfile routes no
name to it) — **Backend:** ROCm GPU (gfx1151) — **Docs:**
https://github.com/Comfy-Org/ComfyUI
## Facts ## Facts
@@ -12,11 +12,12 @@ timestamp: 2026-08-26
# Dankclaw Stack # Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, and development stack. It Dankclaw (192.168.0.143) runs the AI, search, development, and observability
hosts the LLM inference endpoint (`frank-default-large`, `:7770`, CIRU stack. Its LLM inference endpoint `frank-large` (`:7770`) is a ROCmFPX
ROCmFPX llama.cpp), the agent tools, the privacy search, and the llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
observability stack. The llama.cpp endpoint (`frank-default-small`, the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
`:7770`) runs on Dankserver (192.168.0.235). FLM v1.0.1) serves sight+sound models. The completion endpoint
`frank-small` (FreeToken, `:7770`) runs on Dankserver (192.168.0.235).
```mermaid ```mermaid
graph LR graph LR
@@ -29,29 +30,36 @@ graph LR
S --> V[Gluetun VPN] S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770] O --> L[llama.cpp dankserver :7770]
O --> H[Hermes] O --> H[Hermes]
O --> E[CIRU llama.cpp :7770] O --> E[frank-large :7770]
``` ```
## Services ## Services
| Service | Hostname | Port | Docs | | Service | Vhost (guard status) | Port/route | Docs |
|---------|----------|------|------| |---------|----------------------|------------|------|
| SearXNG | search.dankclaw.local | 8081 | [[SearXNG]] | | SearXNG | `search.franksdank.cloud` (guarded: Anubis → Authentik) | published by gluetun at `8081:8080`; searxng itself has no published port | [[SearXNG]] |
| ComfyUI | comfyui.dankclaw.local | 8188 | [[ComfyUI]] | | ComfyUI | none — LAN-only by construction | `8188` | [[ComfyUI]] |
| OpenWebUI | chat.dankclaw.local | 3005 | [[OpenWebUI]] | | OpenWebUI | `ai.franksdank.cloud` (exempt — own auth) | `3005 → 8080` | [[OpenWebUI]] |
| Hermes | hermes.dankclaw.local | 8090 | [[Hermes]] | | Hermes | `hermes.franksdank.cloud` (exempt by construction; internal-only via DNS absence) | webui `8787`; gateway publishes none | [[Hermes]] |
| Gitea | gitea.dankclaw.local | 3000 | [[Gitea]] | | Gitea | `gitea.franksdank.cloud` (exempt — git smart HTTP) | `3000`, SSH `2222` | [[Gitea]] |
| Grafana | grafana.dankclaw.local | 3002 | [[Grafana]] | | Grafana | `grafana.franksdank.cloud` (guarded) | Caddy routes post-auth to `grafana:3000` | [[Grafana]] |
| Prometheus | prometheus.dankclaw.local | 9090 | [[Prometheus]] | | Prometheus | `prometheus.franksdank.cloud` (guarded) | `9090` | [[Prometheus]] |
| Homepage | dankclaw.local | 80/443 | — | | Loki | `loki.franksdank.cloud` (guarded; dedicated loki-proxy outpost provider with `internal_host = caddy:8017`) | `3100` | [[Observability]] |
| frank-default-large (host service, Dankclaw) | — | 7770 | [[deployment]] | | Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
| frank-default-small (host service, Dankserver) | — | 7770 | [[deployment]] | | Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
| frank-large (host service) | — | `7770` | [[deployment]] |
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
| frank-small (host service, Dankserver) | — | `7770` | [[deployment]] |
The last two rows are bare-metal systemd services, not containers. This The LLM rows are bare-metal systemd services, not containers. This
repo owns their scripts, units, and knowledge base (see repo owns their scripts, units, and knowledge base (see
[serverAppConfigs-docs/LLM/index.md](../LLM/index.md)). Current mapping (2026-08-22): [LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
`frank-default-large` runs the CIRU ROCmFPX llama.cpp on Dankclaw; `frank-large` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
`frank-default-small` runs the upstream llama.cpp on Dankserver. on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
`frank-small` runs FreeToken v0.1.2 serving
Huihui-Qwen3.6-35B-A3B-abliterated-FP8 on Dankserver.
## Boot Persistence ## Boot Persistence
+20 -8
View File
@@ -63,9 +63,16 @@ The containers use the quadlet fresh-start model. A stop removes the container (
## Secrets ## Secrets
The database password lives in `/opt/gitea/.env` on the host (mode 0600). The quadlet files reference it with `EnvironmentFile=`. The password never enters this repository. Placeholders live in [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example). The database password lives in `/home/dank/Containers/gitea/env` on the host
(mode 0600); `gitea-postgres.container` and `gitea-server.container` load it
via `EnvironmentFile=` (verified from the live units — the `${GITEA_DB_PASSWORD}`
placeholders are expanded by systemd at start). The password never enters this
repository. Placeholders live in
[`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example).
Do not change the password in place. Change the PostgreSQL role first, then update `/opt/gitea/.env` and the `[database] PASSWD` line in the app.ini (see Gotchas), then restart the stack. Do not change the password in place. Change the PostgreSQL role first, then
update the env file and the `[database] PASSWD` line in the app.ini (see
Gotchas), then restart the stack.
## Configuration files ## Configuration files
@@ -73,23 +80,28 @@ Do not change the password in place. Change the PostgreSQL role first, then upda
|------|----------|-----------| |------|----------|-----------|
| Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](../../dankclaw/gitea/quadlet/) | | Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](../../dankclaw/gitea/quadlet/) |
| Caddyfile | `/home/dank/Containers/caddy/Caddyfile` | [`dankclaw/Caddyfile`](../../dankclaw/Caddyfile) | | Caddyfile | `/home/dank/Containers/caddy/Caddyfile` | [`dankclaw/Caddyfile`](../../dankclaw/Caddyfile) |
| Secrets | `/opt/gitea/.env` (0600) | [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example) | | Secrets | `/home/dank/Containers/gitea/env` (0600; loaded via `EnvironmentFile=`) | [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example) |
After editing a repo copy, deploy through `scripts/deploy.sh dankclaw`. The Gitea database/server units are installed from this directory; the canonical Caddy and its rendered Caddyfile are deployed as the separate `caddy` service. Do not manually recreate the retired `gitea-caddy` unit. After editing a repo copy, deploy through `scripts/deploy.sh dankclaw`. The Gitea database/server units are installed from this directory; the canonical Caddy and its rendered Caddyfile are deployed as the separate `caddy` service. Do not manually recreate the retired `gitea-caddy` unit.
## Data volumes ## Data volumes
Per the live quadlet units (`Volume=` lines):
| Container path | Host path | Contents | | Container path | Host path | Contents |
|----------------|-----------|----------| |----------------|-----------|----------|
| `/data` | `/opt/gitea/data/gitea-gitea` | Git repositories, app.ini, attachments | | `/data` | `/home/dank/Containers/gitea/data` | Git repositories, app.ini, attachments |
| `/var/lib/postgresql/data` | `/opt/gitea/data/gitea-postgres` | PostgreSQL data | | `/var/lib/postgresql/data` | `/home/dank/Containers/gitea/postgres` | PostgreSQL data |
Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge base page [[Gitea]]. Back up `/home/dank/Containers/gitea/` (tar) or `pg_dump` the database. See the knowledge base page [[Gitea]].
## Gotchas ## Gotchas
- **app.ini wins over env vars.** Gitea writes `/data/gitea/conf/app.ini` (host: `/opt/gitea/data/gitea-gitea/gitea/conf/app.ini`) on first start. The `[database] HOST` and `PASSWD` lines persist and override the `DATABASE_HOST`/`DATABASE_PASSWD` env vars. When you rename the database container, edit the `HOST` line. - **Data files are owned by `dank` (uid 1000), not root.** The quadlet units run
- **The files in the data directory are owned by host UID 100999**, not `dank` (rootless ID mapping: container uid 1000 → host 100999). Edit them inside the container with `podman exec --user 1000 gitea-server ...`. podman in the user manager with no `--user`/UserNS override, so container uid
1000 is literally dank's host uid — there is no subuid offset for these
containers. Edit files as the owning user with
`podman exec --user 1000 gitea-server ...`.
- **The host auto-reboots.** `unattended-upgrades` reboots Dankclaw when a package needs it. The quadlet units make Gitea survive that. See the 2026-08-16 incident below. - **The host auto-reboots.** `unattended-upgrades` reboots Dankclaw when a package needs it. The quadlet units make Gitea survive that. See the 2026-08-16 incident below.
- **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../../dankclaw/compose.yaml) maps 3001. Align the two before deploying the full stack. - **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../../dankclaw/compose.yaml) maps 3001. Align the two before deploying the full stack.
- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the Quadlet units on ports 3000 and 2222. - **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the Quadlet units on ports 3000 and 2222.
+8 -4
View File
@@ -2,7 +2,11 @@
Hermes is the self-improving AI agent from Nous Research. It runs in a terminal, Telegram, Discord, Slack, WhatsApp, or Signal, and keeps a persistent learning loop: it creates skills from experience, searches past conversations, and builds a model of the user across sessions. Hermes is the self-improving AI agent from Nous Research. It runs in a terminal, Telegram, Discord, Slack, WhatsApp, or Signal, and keeps a persistent learning loop: it creates skills from experience, searches past conversations, and builds a model of the user across sessions.
**Host:** Dankclaw — **Hostname:** `hermes.dankclaw.local` — **Docs:** https://hermes-agent.nousresearch.com/docs **Host:** Dankclaw — the gateway itself publishes **no port** (it runs in the
host network namespace and dials the Signal daemon at `127.0.0.1:8080`); the
user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached via
`hermes.franksdank.cloud` — internal-only by DNS absence (no public Porkbun A
record) — **Docs:** https://hermes-agent.nousresearch.com/docs
**Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted **Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted
at `/data` in all three Hermes containers (gateway, webui, dashboard). Each at `/data` in all three Hermes containers (gateway, webui, dashboard). Each
@@ -33,9 +37,9 @@ is not durable. The tracked, sanitized configuration baseline is
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN. `0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
- The data volume persists skills, memory, and session history. - The data volume persists skills, memory, and session history.
- Hermes has two named local providers: `dankclaw-large` reaches - Hermes has two named local providers: `dankclaw-large` reaches
`http://192.168.0.143:7770/v1` and exposes `frank-default-large`; `http://192.168.0.143:7770/v1` and exposes `frank-large`;
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes `dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
`frank-default-small`. `frank-small`.
- The live `config.yaml` contains secrets and user state and is not committed; - The live `config.yaml` contains secrets and user state and is not committed;
the tracked example is the sanitized backup of its non-secret settings. the tracked example is the sanitized backup of its non-secret settings.
@@ -50,7 +54,7 @@ is not durable. The tracked, sanitized configuration baseline is
## Gotchas ## Gotchas
- Hermes is under active development; pin the image tag for reproducibility. - Hermes is under active development; pin the image tag for reproducibility.
- Model choice drives quality; the local `frank-default-large` (Dankclaw, 7770) is the intended backend. - Model choice drives quality; the local `frank-large` (Dankclaw, 7770) is the intended backend.
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth. - Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
## Browser automation (Chrome DevTools Protocol daemon) ## Browser automation (Chrome DevTools Protocol daemon)
+20 -10
View File
@@ -2,32 +2,42 @@
OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compatible endpoints (vLLM, llama.cpp, Ollama) and adds RAG, plugins, tool calling, and multi-user support. OpenWebUI is a self-hosted chat interface for LLMs. It connects to OpenAI-compatible endpoints (vLLM, llama.cpp, Ollama) and adds RAG, plugins, tool calling, and multi-user support.
**Host:** Dankclaw — **Port:** 3005 (host) → 8080 (container) — **Boot-persistent:** Quadlet — **Docs:** https://github.com/open-webui/open-webui **Host:** Dankclaw — **Port:** 3005 (host) → 8080 (container) — **Vhost:**
`ai.franksdank.cloud` routes straight to `192.168.0.143:3005` (unguarded —
OpenWebUI enforces its own auth) — **Boot-persistent:** Quadlet — **Docs:**
https://github.com/open-webui/open-webui
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `ghcr.io/open-webui/open-webui:main` (CPU build; Dankclaw is CPU-only) | | Image | `ghcr.io/open-webui/open-webui:main` (verified from the live quadlet unit) |
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. | | Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` | | Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
| Default backend | CIRU llama.cpp on Dankclaw `:7770` = `frank-default-large` (via host LAN IP `192.168.0.143:7770`) | | Default backend | `frank-large` on Dankclaw `:7770` (ROCmFPX llama.cpp fork — see the LLM knowledge base below) — reached via host LAN IP `192.168.0.143:7770` |
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) | | Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` | | Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` | | Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
## The LLM backends ## The LLM backends
The fleet runs two OpenAI-compatible endpoints. This mapping is the **live** state (verified 2026-08-22). Both services listen on `:7770`, distinguished by host. On 2026-08-22 `frank-default-large` moved from `:52625` to `:7770`. The fleet runs two OpenAI-compatible endpoints. This mapping is the **live**
state (verified 2026-09-03). Both services listen on `:7770`, distinguished by
host; `frank-large` moved from `:52625` to `:7770` on 2026-08-22.
| Endpoint | Model name | Host | Capability | | Endpoint | Model name | Host | Capability |
|----------|-----------|------|-----------| |----------|-----------|------|------------|
| `:7770` (CIRU llama.cpp) | `frank-default-large` | Dankclaw | completion | | `:7770` (ROCmFPX llama.cpp fork) | `frank-large` | Dankclaw | completion |
| `:7770` (llama.cpp) | `frank-default-small` | Dankserver | completion, multimodal | | `:7770` (FreeToken v0.1.2) | `frank-small` | Dankserver | completion, multimodal |
The default connection points at the local vLLM (`frank-default-large`), so chat stays on-host. To add the other backend, open **Admin → Connections** in the UI and point an OpenAI connection at `http://192.168.0.235:7770/v1` (key `none`). The default connection points at the on-host endpoint (`frank-large`).
To add the other backend, open **Admin → Connections** in the UI and point an
OpenAI connection at `http://192.168.0.235:7770/v1` (key `none`).
> Both endpoints are bare-metal systemd services managed by this repository (the former `vllmScripts` repo, merged in 2026-08-22). Model swaps, context sizing, and spec tuning happen here. See the [LLM knowledge base](../LLM/index.md). > Both endpoints are bare-metal systemd services managed by this repository
(the former `vllmScripts` repo, merged in 2026-08-22). Model swaps, context
sizing, and spec tuning happen there. See the [LLM knowledge
base](../LLM/index.md).
## Boot persistence (Quadlet) ## Boot persistence (Quadlet)
@@ -58,7 +68,7 @@ The quadlet carries a built-in healthcheck (30s interval, 3 retries, 300s start
## Gotchas ## Gotchas
- Use the non-CUDA image on this CPU-only host. - The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
- The default admin account is created on first login; secure it immediately. - The default admin account is created on first login; secure it immediately.
- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready. - If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
- The LLM is reached via the host LAN IP `192.168.0.143:7770`, not the bridge gateway: the gateway (`10.89.3.1`) lives in the rootless netns, while llama-server runs in the host netns, so the gateway refuses. `host.containers.internal` is slirp4netns-only and does not resolve on a bridge either. To reach the Dankserver backend, use the LAN address `192.168.0.235:7770`. - The LLM is reached via the host LAN IP `192.168.0.143:7770`, not the bridge gateway: the gateway (`10.89.3.1`) lives in the rootless netns, while llama-server runs in the host netns, so the gateway refuses. `host.containers.internal` is slirp4netns-only and does not resolve on a bridge either. To reach the Dankserver backend, use the LAN address `192.168.0.235:7770`.
+7 -3
View File
@@ -10,7 +10,7 @@ PostgreSQL is the database backend for Gitea. It runs on the same podman network
|------|-------| |------|-------|
| Image | `docker.io/library/postgres:16-alpine` | | Image | `docker.io/library/postgres:16-alpine` |
| Port | 5432 (internal only) | | Port | 5432 (internal only) |
| Data volume | bind mount at `/var/lib/postgresql/data` | | Data volume | `/home/dank/Containers/gitea/postgres:/var/lib/postgresql/data` (live bind, verified from the unit) |
## Configuration ## Configuration
@@ -42,9 +42,13 @@ environment:
The live `app.ini` on the host overrides env vars. When you rename the database container, update its `[database] HOST` line. See [[Gitea]] Gotchas. The live `app.ini` on the host overrides env vars. When you rename the database container, update its `[database] HOST` line. See [[Gitea]] Gotchas.
## Migration ## Layout (current)
The live Gitea stack on Dankclaw already runs this image with the data at `/opt/gitea/data/gitea-postgres`. The migration keeps the bind mount in place; see [[Gitea]]. The live `gitea-postgres` unit mounts its data at
`/home/dank/Containers/gitea/postgres:/var/lib/postgresql/data` — the old
`/opt/gitea/data/gitea-postgres` compose-era path is gone. Secrets come from
the host-local env file `/home/dank/Containers/gitea/env` (0600) via
`EnvironmentFile=`; `POSTGRES_PASSWORD` never enters this repo. See [[Gitea]].
## Healthcheck ## Healthcheck
+16 -6
View File
@@ -30,14 +30,19 @@ Port publishing happens on the **gluetun** unit, not on searxng. See [[Gluetun]]
The trio runs as quadlet units, not podman-compose, so it starts at boot. The unit files live in `searxng/quadlet/` and install to `~/.config/containers/systemd/`. The trio runs as quadlet units, not podman-compose, so it starts at boot. The unit files live in `searxng/quadlet/` and install to `~/.config/containers/systemd/`.
The units live in `dankclaw/searxng/quadlet/` and install to
`~/.config/containers/systemd/` on Dankclaw (verified live):
| Unit file | Service | Role | | Unit file | Service | Role |
|-----------|---------|------| |-----------|---------|------|
| `searxng.network` | `searxng-network.service` | DNS-enabled bridge, subnet pinned to 10.89.2.0/24 | | `searxng.network` | generated network service | DNS-enabled bridge; the created network is named `systemd-searxng`, subnet pinned to 10.89.2.0/24 |
| `gluetun.container` | `gluetun.service` | WireGuard tunnel; publishes 8081:8080 | | `gluetun.container` | `gluetun.service` | WireGuard tunnel (env-file secrets); publishes `8081:8080` — the only published port in the trio |
| `searxng-valkey.container` | `searxng-valkey.service` | Valkey cache, alias `valkey` | | `searxng-valkey.container` | `searxng-valkey.service` | Valkey cache on the searxng network, alias `valkey` (see [[Valkey]]) |
| `searxng.container` | `searxng.service` | Reuses gluetun's netns; publishes no port | | `searxng.container` | `searxng.service` | `Network=gluetun.container` — joins the tunnel's netns, so it publishes no port of its own; reached through gluetun's 8081 |
gluetun and valkey join the searxng network. searxng reuses gluetun's netns and reaches valkey by name on that network. Each unit sets `WantedBy=default.target`. User linger is on, so the units start at boot. gluetun and valkey join the searxng network. searxng reuses gluetun's netns
and reaches valkey by name on that network. Each unit sets
`WantedBy=default.target`. User linger is on, so the units start at boot.
Deploy with `scripts/deploy-searxng-quadlet.sh dankclaw`. It copies the units, daemon-reloads, enables them, and starts them in dependency order. Deploy with `scripts/deploy-searxng-quadlet.sh dankclaw`. It copies the units, daemon-reloads, enables them, and starts them in dependency order.
@@ -112,7 +117,12 @@ Why so many ranges: rootless podman strips the client IP on published-port hits.
The rate limiter's JSON cap is `API_MAX = 4` requests per IP per hour. The constant is hardcoded in Python (`searx/botdetection/ip_limit.py`). It cannot be raised from `limiter.toml`. `pass_ip` is the only bypass. This is what lets 4 parallel subagents poll the JSON API at full rate without a 429. The rate limiter's JSON cap is `API_MAX = 4` requests per IP per hour. The constant is hardcoded in Python (`searx/botdetection/ip_limit.py`). It cannot be raised from `limiter.toml`. `pass_ip` is the only bypass. This is what lets 4 parallel subagents poll the JSON API at full rate without a 429.
Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local subagent can hit the published port directly because its slirp source is now trusted. The searxng subnet 10.89.2.0/24 is in `trusted_proxies`, so Caddy's `X-Forwarded-For` is honored. The subnet is pinned in `searxng.network`, so it stays stable across reboots. Reach SearXNG through Caddy (`search.franksdank.cloud`, guarded) for browser
use. A local subagent can hit the published port directly because its
slirp/loopback/LAN sources are in `pass_ip`. The searxng network
(`systemd-searxng`, 10.89.2.0/24) is in `trusted_proxies`, so Caddy's
`X-Forwarded-For` is honored. The subnet is pinned in `searxng.network`, so it
stays stable across reboots.
## Unblocking Google & Search Providers ## Unblocking Google & Search Providers
+84 -369
View File
@@ -1,8 +1,24 @@
---
type: reference
title: *arr Stack
description: The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) on FrankBook-Pro — topology, wiring, auth posture, and gotchas.
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# *arr Stack # *arr Stack
The *arr suite is a collection of automated media management tools for Usenet and BitTorrent. For Dankserver — which runs Copyparty, Calibre-Web, RomM, and Jellyfin — the *arr stack provides automatic downloading and library management that Copyparty alone does not cover: quality profile enforcement, series/movie discovery, metadata enrichment, and automated file organization. The *arr suite is a collection of automated media management tools for Usenet
and BitTorrent. The stack runs on **FrankBook-Pro** — the Mac mini that fronts
this household's downloads — as quadlet units inside its podman machine, not on
Dankserver. It provides automatic downloading and library management that
Copyparty alone does not cover: quality profile enforcement, series/movie
discovery, metadata enrichment, and automated file organization.
The core chain is: **Prowlarr** (indexer API aggregator) → **Sonarr/Radarr/Lidarr/Readarr** (media-specific managers) → **qBittorrent** (download client) → **Jellyfin** (media server scan → auto-refresh). The core chain is: **Prowlarr** (indexer API aggregator) →
**Sonarr/Radarr/Lidarr/Readarr** (media-specific managers) → **qBittorrent**
(download client, sharing the [[Gluetun]] WireGuard tunnel netns) →
**Jellyfin** (media server scan → auto-refresh).
## Service Overview ## Service Overview
@@ -12,334 +28,59 @@ The core chain is: **Prowlarr** (indexer API aggregator) → **Sonarr/Radarr/Lid
| Radarr | Movie management | `lscr.io/linuxserver/radarr:latest` | 7878 | 512 MB | | Radarr | Movie management | `lscr.io/linuxserver/radarr:latest` | 7878 | 512 MB |
| Sonarr | TV show management | `lscr.io/linuxserver/sonarr:latest` | 8989 | 512 MB | | Sonarr | TV show management | `lscr.io/linuxserver/sonarr:latest` | 8989 | 512 MB |
| Lidarr | Music management | `lscr.io/linuxserver/lidarr:latest` | 8686 | 256 MB | | Lidarr | Music management | `lscr.io/linuxserver/lidarr:latest` | 8686 | 256 MB |
| Readarr | Ebook/audiobook management | `lscr.io/linuxserver/readarr:latest` | 8787 | 256 MB | | Readarr | Ebook/audiobook management | `lscr.io/linuxserver/readarr:amd64-1.0.980` (pinned — LSIO ships no untagged `latest` for readarr) | 8787 | 256 MB |
| qBittorrent | Torrent download client | `lscr.io/linuxserver/qbittorrent:latest` | 8080 | 512 MB | | qBittorrent | Torrent download client | `lscr.io/linuxserver/qbittorrent:4.6.7` (pinned; see [[qBittorrent]]) | 8080 (published by the gluetun unit) | 512 MB |
| Jackett | Indexer proxy (optional) | `lscr.io/linuxserver/jackett:latest` | 9117 | 256 MB |
| Whisparr | NSFW movie management (optional) | `lscr.io/linuxserver/whisparr:latest` | 6969 | 256 MB |
**Minimum total RAM:** ~3 GB for full stack (headroom recommended). Jackett and Whisparr were evaluated as optional extras but never deployed
(no unit, no vhost, no live container).
## Architecture ## Deployment shape (current)
```mermaid Each service is a quadlet `.container` unit in this repo under
graph LR `frankbook-pro/<svc>/quadlet/`, installed inside the podman machine at
I[Indexers<br/>Torrents, Usenet] --> P[Prowlarr<br/>Indexer Manager] `/home/core/.config/containers/systemd/<svc>.service` by the frankbook-pro
P --> S[Sonarr<br/>TV] handler in `scripts/deploy-remote.sh`. Every unit runs with
P --> R[Radarr<br/>Movies] `PUID=0 PGID=0 TZ=America/Los_Angeles UMASK_SET=022`, mounts its config root at
P --> L[Lidarr<br/>Music] `/Volumes/COMPLETE/<svc>-data:/config`, publishes its API port on the host, and
P --> Rd[Readarr<br/>Books] runs `Restart=always` with `TimeoutStartSec=900`.
S --> Q[qBittorrent]
R --> Q
L --> Q
Rd --> Q
Q --> M1[/tv/]
Q --> M2[/movies/]
Q --> M3[/music/]
Q --> M4[/books/]
M1 --> J[Jellyfin]
M2 --> J
M3 --> J
M4 --> CW[Calibre-Web]
```
## Docker Compose qBittorrent is special: it shares the [[Gluetun]] container's network namespace
(`Network=gluetun.container`) so torrent egress never touches the residential
IP. A netns-sharer publishes no ports of its own — qB's WebUI (8080) and
BitTorrent port (6881 TCP+UDP) are published on the gluetun unit instead.
```yaml ## Historical design-era compose (retired)
networks:
appnet:
driver: bridge
services: The original design deployed the stack on Dankserver via
prowlarr: `dankserver/compose.yaml` with `/opt/<svc>/config` data roots and
image: lscr.io/linuxserver/prowlarr:latest `TZ=America/New_York`. That layout is retired; `dankserver/compose.yaml` is
container_name: prowlarr kept as reference only. Reality (unit files above) wins: data roots are the
restart: unless-stopped Mac's `/Volumes/COMPLETE` (completed media) and `/Volumes/INCOMPLETE`
environment: (in-flight downloads) volumes, mounted into qB at identical paths, and every
- PUID=1000 unit runs `TZ=America/Los_Angeles`.
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/prowlarr/config:/config
- /mnt/media/downloads:/downloads:ro
ports:
- "9696:9696"
networks: [appnet]
radarr: ## Access / auth posture
image: lscr.io/linuxserver/radarr:latest
container_name: radarr
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/radarr/config:/config
- /mnt/media/movies:/movies
- /mnt/media/downloads:/downloads
ports:
- "7878:7878"
networks: [appnet]
sonarr: There is **no Caddy vhost for any arr app** — the *arr stack is exempt by
image: lscr.io/linuxserver/sonarr:latest construction from the Anubis guard chain (its own auth). The stack is reached
container_name: sonarr directly by IP:port on the LAN (`http://<frankbook-pro>:8989` etc.).
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/sonarr/config:/config
- /mnt/media/tv:/tv
- /mnt/media/downloads:/downloads
ports:
- "8989:8989"
networks: [appnet]
lidarr: Auth-config history (verified live 2026-08-29): all five arrs shipped with
image: lscr.io/linuxserver/lidarr:latest `AuthenticationMethod=None` **plus** `AuthenticationRequired=Enabled`. On the
container_name: lidarr v1 (.NET) apps that combo is self-contradictory — UI Save is permanently
restart: unless-stopped disabled and any settings save throws `DryIoc.ScopedItemException` inside
environment: `AuthorizationMiddleware` (500 sprawl on every page after restart). The valid
- PUID=1000 representation of "no auth" in v1 is **absence of the `<AuthenticationRequired>`
- PGID=1000 key**, not the literal string `Disabled` — readarr's config (key absent) booted
- TZ=America/New_York clean while its three siblings (key=`Disabled`) 500-sprawled, which is how it
- UMASK_SET=022 was caught. Fix applied live: key deleted on sonarr/radarr/lidarr/prowlarr,
volumes: containers restarted, all five verified serving. Posture = zero-auth arrs on
- /opt/lidarr/config:/config LAN and WAN; the Anubis wall guards homepage/search/hermes/prometheus/romm/files
- /mnt/media/music:/music but **not** the arrs — first-account-wins is the current race condition if that
- /mnt/media/downloads:/downloads ever flips to `External` auth (needs an IdP first; auth method choice remains an
ports: operator decision).
- "8686:8686"
networks: [appnet]
readarr: ## Chain wiring (verified live 2026-08-30)
image: lscr.io/linuxserver/readarr:latest
container_name: readarr
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/readarr/config:/config
- /mnt/media/books:/books
- /mnt/media/downloads:/downloads
ports:
- "8787:8787"
networks: [appnet]
qbittorrent:
image: lscr.io/linuxserver/qbittorrent:latest
container_name: qbittorrent
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/qbittorrent/config:/config
- /mnt/media/downloads:/downloads
- /mnt/media/downloads/incomplete:/downloads/incomplete
ports:
- "8080:8080"
networks: [appnet]
# Optional: Jackett for niche indexers
jackett:
image: lscr.io/linuxserver/jackett:latest
container_name: jackett
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/jackett/config:/config
- /opt/jackett/torrences:/torrences
ports:
- "9117:9117"
networks: [appnet]
# Optional: Whisparr for adult content
whisparr:
image: lscr.io/linuxserver/whisparr:latest
container_name: whisparr
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/whisparr/config:/config
- /mnt/media/whisparr-movies:/whisparr-movies
- /mnt/media/downloads:/downloads
ports:
- "6969:6969"
networks: [appnet]
```
## Caddy Configuration
```caddy
radarr.dankserver.local {
reverse_proxy localhost:7878
}
sonarr.dankserver.local {
reverse_proxy localhost:8989
}
lidarr.dankserver.local {
reverse_proxy localhost:8686
}
readarr.dankserver.local {
reverse_proxy localhost:8787
}
prowlarr.dankserver.local {
reverse_proxy localhost:9696
}
qbittorrent.dankserver.local {
reverse_proxy localhost:8080
}
# Optional: Jackett
jackett.dankserver.local {
reverse_proxy localhost:9117
}
# Optional: Whisparr
whisparr.dankserver.local {
reverse_proxy localhost:6969
}
```
## Configuration Steps
### Phase 1: Download Client
1. Deploy qBittorrent first (or Transmission).
2. In the qBittorrent Web UI, set **No auth mode** for LAN (Settings → Web UI).
3. Create download categories: `movies`, `tv`, `music`, `books`.
4. Change the default Web UI password.
### Phase 2: Indexer Management
1. Deploy Prowlarr.
2. Add indexers (3–5 recommended for testing). Use the **Test** button on each.
3. Add qBittorrent as a download client in Prowlarr.
4. Verify search from the Prowlarr web UI.
### Phase 3: Arr Apps (one at a time)
1. Deploy Sonarr first (TV shows are the most common use case).
2. Add Prowlarr as the indexer source (Settings → Indexers → Add Prowlarr).
3. Add qBittorrent as the download client with categories.
4. Add a test series. Wait for the episode to download. Verify in `/tv/`.
5. Repeat for Radarr, Lidarr, Readarr.
### Phase 4: Media Server Integration
1. Configure Jellyfin libraries pointing to `/movies`, `/tv`, `/music`, `/books`.
2. Set **On Download** notifications in each arr app → Jellyfin webhook URL.
3. Test end-to-end: Add a movie in Radarr → download → verify in Jellyfin.
## Quality Profiles
Each arr app uses a hierarchical quality system:
1. **Quality Profile** — Defines tiers (e.g., 2160p → 1080p BluRay → 720p).
2. **Cutoff** — The quality target. Downloads at or above cutoff are accepted.
3. **Custom Formats** — Additional filters (e.g., "Has HDR", "Is Director's Cut").
4. **Search on Grab** — When content becomes available, all indexers are queried.
### Radarr (Movies)
```
Profile: Movies 4K
Tiers:
1. 2160p BluRay REMUX
2. 2160p BluRay
3. 2160p WEB-DL
4. 1080p BluRay
5. 1080p WEB-DL
6. 720p
Cutoff: 1080p BluRay
```
### Sonarr (TV)
```
Profile: TV - Standard
Tiers: 2160p → 1080p → 720p
Min Size: 500 MB
Max Size: 50 GB
Cutoff: 1080p
Language Profile: English audio preferred
```
### Lidarr (Music)
```
Profile: Music - Standard
Tiers: Lossless (FLAC) → 320 kbps → 192 kbps
Cutoff: 320 kbps (or Lossless)
```
## Metadata Sources
| App | Primary Source | Secondary |
|-----|---------------|-----------|
| Radarr | TMDB | TheMovieDB API |
| Sonarr | TMDB | TVDB |
| Lidarr | MusicBrainz | — |
| Readarr | Goodreads, StoryGraph | — |
| Whisparr | XH | TMDB |
## *arr Stack vs Copyparty
| Feature | Copyparty | *arr Stack |
|---------|-----------|------------|
| Auto-download from torrents | Basic (RSS + scripts) | Native, integrated |
| Quality profiles | No | Multi-tier, custom formats |
| Series/movie tracking | No | Watch lists, air dates |
| Manual library management | Yes | Yes, metadata-driven |
| Multi-user watchlists | No | Yes |
| Complexity | Low | Medium |
| Resource usage | Minimal | Moderate |
Use Copyparty for curated file collections. Use *arr for automatic downloads. Use both together for maximum functionality — point *arr download directories at Copyparty's watched folders.
## Live State (as-built, verified 2026-08-29)
Deployed shape differs from the design compose above — reality wins:
- Mounts are the real array paths (`/mnt/entropy/downloads`, case-exact dirs
like `Movies`/`TVShows`), not `/mnt/media/*`; TZ is `America/Los_Angeles`.
- Readarr runs `lscr.io/linuxserver/readarr:amd64-1.0.980` — LSIO ships no
untagged `latest` for readarr; the tag was pinned after a manifest probe.
- Prowlarr's `/config` mount was missing mid-deploy (off-by-one in an edit);
restored in commit `67695cf`.
- **Auth posture: every arr runs `AuthenticationMethod=None` +
`AuthenticationRequired=Enabled`** — verified live via each config.xml and by
probing `/api/v3/system/status` unauthenticated (401 on all five, local and
through the vhost). Net effect: a login screen with **zero users defined**;
whoever creates the first account owns the instance. Decide the auth method
(`None` behind the Anubis wall vs `External` via authentik) *before* WAN
exposure — first-account-wins is the current race condition.
- API keys live in each `/config/config.xml` (`<ApiKey>`); they are local-only
secrets and stop being secret the moment a vhost serves the API unauthenticated.
### Chain wiring (verified live 2026-08-29)
The download chain is wired and test-first verified: **indexers → Prowlarr → The download chain is wired and test-first verified: **indexers → Prowlarr →
per-arr qBittorrent clients (category = app name) → import**. per-arr qBittorrent clients (category = app name) → import**.
@@ -364,72 +105,46 @@ per-arr qBittorrent clients (category = app name) → import**.
and hand off to their own qB download client (category routing). Mappings and hand off to their own qB download client (category routing). Mappings
materialize at grab time — zero mappings right after wiring is expected, materialize at grab time — zero mappings right after wiring is expected,
not a fault. not a fault.
**Chain complete (same day):** all four arr roots live — Sonarr `/tv`,
Radarr `/movies` (201), and Lidarr `/music` + Readarr `/books` created via
API after the v1 validator layers peeled back one at a time
(`defaultQualityProfileId`/`defaultMetadataProfileId` must reference an
*existing* profile id on v1 apps; v1 has no `/api/v1/profile` endpoint —
probe endpoints by status code, empty body = absent).
**Auth-config pitfall (found same day):** all five arrs shipped with ## Disk layout (current)
`AuthenticationMethod=None` **plus** `AuthenticationRequired=Enabled`. On the
v1 (.NET) apps that combo is self-contradictory — UI Save is permanently
disabled and any settings save throws
`DryIoc.ScopedItemException` inside `AuthorizationMiddleware` (500 sprawl on
every page after restart). The valid representation of "no auth" in v1 is
**absence of the `<AuthenticationRequired>` key**, not the literal string
`Disabled` — readarr's config (key absent) booted clean while its three
siblings (key=`Disabled`) 500-sprawled, which is how it was caught. Fix
applied live: key deleted on sonarr/radarr/lidarr/prowlarr, containers
restarted, all five verified serving. Posture = zero-auth arrs on LAN and
WAN; the Anubis wall guards homepage/search/hermes/prometheus/romm/files
but **not** the arrs — if WAN exposure of `/sonarr`, `/radarr` etc. is
undesired, guard them in the Anubis policy or flip auth to `External`
(needs an IdP first; auth method choice remains an operator decision).
## Disk Layout
``` ```
/mnt/media/ /Volumes/COMPLETE <- completed-media volume (Mac volume)
├── movies/ → Radarr root ├── sonarr-data:/config <- per-app config roots, one per unit
├── tv/ → Sonarr root ├── radarr-data:/config
├── music/ → Lidarr root ├── lidarr-data:/config
├── books/ → Readarr root ├── readarr-data:/config
├── whisparr-movies/ → Whisparr root ├── qbittorrent-data:/config
├── downloads/ → qBittorrent download dir └── <movie/tv/music/books roots> <- arr import roots live here
│ ├── movies/ /Volumes/INCOMPLETE <- qB in-flight downloads (same path inside qB)
│ ├── tv/
│ ├── music/
│ └── books/
└── incomplete/ → qBittorrent incomplete downloads
``` ```
qB mounts both volumes at their identical host paths
(`Volume=/Volumes/COMPLETE:/Volumes/COMPLETE`,
`Volume=/Volumes/INCOMPLETE:/Volumes/INCOMPLETE`), so a category's download path
is the same string on every container that touches it — no remapping between
download and import.
## Gotchas ## Gotchas
- **Timezone mismatch:** All containers should use the same TZ. Sonarr's scheduler is timezone-sensitive. - **Timezone:** every unit runs `TZ=America/Los_Angeles`; Sonarr's scheduler is timezone-sensitive — keep them uniform.
- **File permissions:** Set `umask=022` in qBittorrent. Ensure arr apps can read/move downloaded files. - **File permissions:** `UMASK_SET=022` on qBittorrent; arr apps run PUID=0 so they can read/move qB downloads.
- **Network conflicts:** *arr apps use many ports. Check for conflicts with existing Dankserver services. - **Network conflicts:** *arr apps use many ports. On the machine VM, only the API ports (8989/7878/8686/8787/9696) and gluetun's published 8080 + 6881 are host-published; netns-sharers (qB) publish none of their own.
- **Prowlarr API key rotation:** If rotated, update it in every arr app. - **Prowlarr API key rotation:** If rotated, update it in every arr app (`Applications` table rows carry the real keys).
- **qBittorrent Web UI password:** Change default on first run. - **qBittorrent WebUI password:** generated at first boot, not a default — persist it via the PBKDF2 conf recipe in [[qBittorrent]] before any restart.
- **Readarr stability:** Readarr is still in development. Do not rely on it for mission-critical collections. - **Readarr stability:** abandoned upstream (retired May 2026); its qB download client is parked pending a successor image. Do not rely on it for mission-critical collections.
- **Whisparr separate Jellyfin library:** Keep adult content separate from regular media.
- **RSS feed in qBittorrent:** Can auto-add torrents matching a pattern. Useful for podcast-like series.
## Best Practices ## Best Practices
1. Use tags to route content to specific download clients or categories. 1. Use tags to route content to specific download clients or categories.
2. Enable the Blocklist in arr apps to prevent re-downloading failed torrents. 2. Enable the Blocklist in arr apps to prevent re-downloading failed torrents.
3. Set download limits in qBittorrent to avoid bandwidth saturation. 3. Set download limits in qBittorrent to avoid bandwidth saturation.
4. Back up arr configs regularly (they are small SQLite DBs + JSON). 4. Back up arr configs regularly (they are small SQLite DBs + JSON) — each lives at `/Volumes/COMPLETE/<svc>-data/config`.
5. Monitor Prowlarr indexer health — broken indexers hide new content. 5. Monitor Prowlarr indexer health — broken indexers hide new content.
6. Start with default quality profiles. Add custom formats gradually. 6. Start with default quality profiles. Add custom formats gradually.
7. Back up arr configs: `./radarr/config`, `./sonarr/config`, etc.
## See Also ## See Also
- [[Dankserver Stack]] - [[Sonarr]] · [[Radarr]] · [[Lidarr]] · [[Readarr]] · [[Prowlarr]] · [[qBittorrent]] — per-service pages
- [[Jellyfin]] — media server that consumes *arr downloads - [[Jellyfin]] — media server that consumes *arr downloads
- [[Copyparty]] — alternative file viewer - [[Copyparty]] — alternative file viewer
- [[Calibre-Web]] — manual ebook management (Readarr auto-downloads into the same library)
- [Sonarr Wiki](https://wiki.servarr.com/sonarr)
- [Radarr Wiki](https://wiki.servarr.com/radarr)
@@ -2,14 +2,16 @@
Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. It serves as the web frontend for the Calibre e-book manager. Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. It serves as the web frontend for the Calibre e-book manager.
**Host:** Dankserver — **Hostname:** `books.dankserver.local` — **Docs:** https://github.com/linuxserver/docker-calibre-web **Host:** Dankserver — **Access:** `books.franksdank.cloud` (unguarded Caddy
vhost — Calibre-Web serves its own login; direct proxy to
`192.168.0.235:8083`) — **Docs:** https://github.com/linuxserver/docker-calibre-web
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Latest version | 0.6.25-ls365 (live); 0.6.27-ls397 upstream | | Running image | `lscr.io/linuxserver/calibre-web:0.6.27-ls397` — **pinned** (verified from the live quadlet unit; the pin was chosen so the cutover changed zero bits of application code) |
| Image | `lscr.io/linuxserver/calibre-web:latest` (live: 0.6.25-ls365) | | Network | `pasta` (the default rootless-podman userslide network); only 8083 is host-published |
| Base OS | Ubuntu Noble (24.04 LTS, since 2024-12-05) | | Base OS | Ubuntu Noble (24.04 LTS, since 2024-12-05) |
| Port | 8083 (host) → 8083 (container) | | Port | 8083 (host) → 8083 (container) |
| Default login | `admin` / `admin123` | | Default login | `admin` / `admin123` |
@@ -34,7 +36,7 @@ Additional packages added: `libxfixes3` and `libasound2t64` to support epub-to-p
|----------|-------|-------------| |----------|-------|-------------|
| `PUID` | `1000` | User ID | | `PUID` | `1000` | User ID |
| `PGID` | `1000` | Group ID | | `PGID` | `1000` | Group ID |
| `TZ` | `Etc/UTC` | Timezone | | `TZ` | `America/Vancouver` (live unit value — not `Etc/UTC`) |
| `DOCKER_MODS` | `linuxserver/mods:universal-calibre` | Optional: enables ebook conversion tools | | `DOCKER_MODS` | `linuxserver/mods:universal-calibre` | Optional: enables ebook conversion tools |
| `OAUTHLIB_RELAX_TOKEN_SCOPE` | `1` | Required for Google OAuth | | `OAUTHLIB_RELAX_TOKEN_SCOPE` | `1` | Required for Google OAuth |
+9 -4
View File
@@ -25,16 +25,21 @@ Calibre 9.2.1 is the version on the live host (9.9.0 is the upstream latest). Ma
## Library Directory ## Library Directory
| Purpose | Host path (convention) | The live Calibre-Web container bind-mounts the library at `/books`:
|---------|------------------------|
| Calibre library | `/opt/calibre-web/books` (bind-mounted to `/books` in the Calibre-Web container) | | Purpose | Host path (live) |
|---------|------------------|
| Calibre library | `/home/dank/Containers/calibre-web/books` → `/books` in the Calibre-Web container |
(The older `/opt/calibre-web/books` path was the compose-era location and is
gone; the quadlet unit mounts the Containers data root.)
## What the Container Needs ## What the Container Needs
- Read/write access to the library directory. Calibre-Web edits metadata through the database file. - Read/write access to the library directory. Calibre-Web edits metadata through the database file.
- Ownership matching `PUID=1000` / `PGID=1000` (the `dank` user on Dankserver). - Ownership matching `PUID=1000` / `PGID=1000` (the `dank` user on Dankserver).
## Optional: Calibre Content Server ## Optional: Calibre Content Server (not running)
Calibre includes a built-in HTTP content server (`calibre-server`) that can serve the library via OPDS feeds and a mobile-friendly web interface. This enables: Calibre includes a built-in HTTP content server (`calibre-server`) that can serve the library via OPDS feeds and a mobile-friendly web interface. This enables:
- OPDS feeds for mobile reading apps (FBReader, Moon+ Reader) - OPDS feeds for mobile reading apps (FBReader, Moon+ Reader)
+16 -13
View File
@@ -2,14 +2,16 @@
Copyparty is a lightweight file server and media streamer that runs as a single Python binary. It handles large payloads, media streaming, resumable uploads, directory browsing, and WebDAV from one process. Copyparty is a lightweight file server and media streamer that runs as a single Python binary. It handles large payloads, media streaming, resumable uploads, directory browsing, and WebDAV from one process.
**Host:** Dankserver — **Hostname:** `files.dankserver.local` — **Docs:** https://github.com/9001/copyparty **Host:** Dankserver — **Access:** `files.franksdank.cloud` guarded by the
Anubis → Authentik chain ([[Caddy]] routes it post-auth to
`192.168.0.235:3923`); the published port itself stays unauthenticated on the
LAN — **Docs:** https://github.com/9001/copyparty
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Latest version | v1.20.21 (2026-08) | | Running image | `ghcr.io/9001/copyparty-ac:latest` (ac = full edition; the bare `copyparty:latest` tag does not exist — ghcr answers 403 on the token request for nonexistent repos). Version at research time (2026-08): v1.20.21 |
| Image | `ghcr.io/9001/copyparty-ac:latest` (ac = full edition; the bare `copyparty:latest` tag does not exist — ghcr answers 403 on the token request for nonexistent repos) |
| Port | 3923 (host) → 3923 (container) | | Port | 3923 (host) → 3923 (container) |
| Volume | bind mount the shared root directory | | Volume | bind mount the shared root directory |
| Auth | Per-directory `.cpy` config files, HTTP basic auth, `dirkeys`, `filekeys` | | Auth | Per-directory `.cpy` config files, HTTP basic auth, `dirkeys`, `filekeys` |
@@ -72,13 +74,12 @@ The `iv` edition is no longer available for arm32/armv6. Use `dj` for maximum co
## Volumes ## Volumes
Live deployment: the whole LUKS array is served at URL `/w` (the ac edition Live deployment: the whole LUKS array is served at URL `/w` via
takes its volume roots from argv plus `chdir: /w` in `/z/initcfg`, not from a `Volume=/mnt/entropy:/w:ro` in the quadlet unit ([[Entropy Volume]]) —
`/data` convention). Access is **read-only since 2026-08-31** (`-v /w::r:ra,dank` read-only **at the mount level**, so no containerized consumer can write into
— see [[#Command-line interface (ac edition)]] above). The compose bind mount is the array through this path regardless of account permissions. The ac edition
`/mnt/entropy:/w` (host path → container path). Note the bind itself is still takes its volume roots from argv (`-v /w::r:ra,dank`), not from a `/data`
rw on disk — copyparty enforces read-only at the application layer, so local convention; see [[#Command-line interface (ac edition)]] above.
services (Jellyfin, Calibre) keep writing into the array directly.
## Docker Compose (reference — retired 2026-09-01; live deployment is the quadlet unit below) ## Docker Compose (reference — retired 2026-09-01; live deployment is the quadlet unit below)
@@ -146,13 +147,15 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
- **Large transfers:** Disable TLS inside the app and let Caddy terminate TLS. This avoids double encryption overhead. - **Large transfers:** Disable TLS inside the app and let Caddy terminate TLS. This avoids double encryption overhead.
- **Dedicated share directory:** Do not give Copyparty the whole disk. Mount only the intended share path. - **Dedicated share directory:** Do not give Copyparty the whole disk. Mount only the intended share path.
- **Whole-disk share (live choice):** this deployment *does* serve the whole - **Whole-disk share (live choice):** this deployment *does* serve the whole
91 T array from `/w`. Since 2026-08-31 the volume is application-level ~91 T array from `/w`. The volume is mounted `:ro` in the unit AND the
read-only for every account, so no remote write path exists. But note: permission flags grant anonymous only `r`, so no remote write path exists at
all now. But note:
**LAN-direct `:3923` is unauthenticated browse/download** (anonymous holds **LAN-direct `:3923` is unauthenticated browse/download** (anonymous holds
`r`) — Caddy + Anubis + authentik guard only `files.franksdank.cloud`, not `r`) — Caddy + Anubis + authentik guard only `files.franksdank.cloud`, not
the published port. Tighten to `-v /w::r,dank` if anonymous LAN reads become the published port. Tighten to `-v /w::r,dank` if anonymous LAN reads become
unacceptable. (Before 2026-08-31, `-v /w::A` made the array world-writable unacceptable. (Before 2026-08-31, `-v /w::A` made the array world-writable
on the LAN — see [[#Command-line interface (ac edition)]].) on the LAN — see [[#Command-line interface (ac edition)]] — and the mount
was rw.)
- **Upload tokens scoped per folder:** Prevents accidental global write access. - **Upload tokens scoped per folder:** Prevents accidental global write access.
- **CSP nonces (v1.20.17+):** Security hardening for JavaScript execution. May break third-party JS plugins (`--js-browser` / `--html-head`). - **CSP nonces (v1.20.17+):** Security hardening for JavaScript execution. May break third-party JS plugins (`--js-browser` / `--html-head`).
- **Custom thumbnail extractors (v1.20.19+):** New `thumbex` framework. Requires configuration in `initcfg`. - **Custom thumbnail extractors (v1.20.19+):** New `thumbex` framework. Requires configuration in `initcfg`.
@@ -1,45 +1,40 @@
# Dankserver Stack # Dankserver Stack
Dankserver (192.168.0.235) runs the media and library stack. Every service is reachable by hostname through the Caddy reverse proxy on the host. Dankserver (192.168.0.235) runs the media and library stack. Public access
goes through the Caddy reverse proxy on Dankclaw; the *arr stack is exempt by
construction — it has no vhost and is reached directly by IP:port on the LAN.
```mermaid ```mermaid
graph LR graph LR
U[User] --> C[Caddy :80/:443] U[User] --> C[Caddy :80/:443 on Dankclaw]
C --> R[RomM] C -->|guarded: Anubis + Authentik| R[RomM :8085]
C --> P[Copyparty] C -->|guarded| P[Copyparty :3923]
C --> W[Calibre-Web] C -->|unguarded| W[Calibre-Web :8083]
C --> J[Jellyfin] C -->|unguarded| J[Jellyfin :8096]
C --> S[Sonarr] C -->|exempt by construction| X[Plex :32400]
C --> D[Radarr] U -->|direct IP:port, no vhost| S[ARR stack + Prowlarr + qBittorrent on FrankBook-Pro]
C --> L[Lidarr]
C --> A[Readarr]
C --> O[Prowlarr]
W --> L2[Calibre library]
``` ```
## Services ## Services
| Service | Hostname | Port | Docs | | Service | Public route | Port | Docs |
|---------|----------|------|------| |---------|--------------|------|------|
| RomM | romm.dankserver.local | 8085 | [[RomM]] | | RomM | `romm.franksdank.cloud` (guarded: Anubis → Authentik; post-auth route → 192.168.0.235:8085) | 8085 | [[RomM]] |
| Copyparty | files.dankserver.local | 3923 | [[Copyparty]] | | Copyparty | `files.franksdank.cloud` (guarded; post-auth route → 192.168.0.235:3923) | 3923 | [[Copyparty]] |
| Calibre-Web | books.dankserver.local | 8083 | [[Calibre-Web]] | | Calibre-Web | `books.franksdank.cloud` (unguarded — own login) | 8083 | [[Calibre-Web]] |
| Jellyfin | jellyfin.dankserver.local | 8096 | [[Jellyfin]] | | Jellyfin | `jelly.franksdank.cloud` (unguarded — own auth) | 8096 | [[Jellyfin]] |
| Sonarr | sonarr.dankserver.local | 8989 | [[ARR-Stack]] | | Plex | `plex.franksdank.cloud` (+ legacy plain-HTTP `plex.franksdank.casa`) | 32400 | [[Plex]] |
| Radarr | radarr.dankserver.local | 7878 | [[ARR-Stack]] |
| Lidarr | lidarr.dankserver.local | 8686 | [[ARR-Stack]] |
| Readarr | readarr.dankserver.local | 8787 | [[ARR-Stack]] |
| Prowlarr | prowlarr.dankserver.local | 9696 | [[ARR-Stack]] |
| qBittorrent | qbittorrent.dankserver.local | 8080 | [[ARR-Stack]] |
## Optional Services The *arr stack (Sonarr 8989, Radarr 7878, Lidarr 8686, Readarr 8787,
Prowlarr 9696) and qBittorrent (8080, behind the [[Gluetun]] tunnel netns)
run on **FrankBook-Pro**, not on this host — see [[ARR-Stack]]; none of them
have a Caddy vhost.
These services are included in the compose file but may be disabled. ## Never-deployed services
| Service | Hostname | Port | Docs | Jackett and Whisparr were evaluated during the ARR-stack design phase and
|---------|----------|------|------| never deployed (no unit, no vhost, no live container). Prowlarr covers the
| Jackett | jackett.dankserver.local | 9117 | [[ARR-Stack]] | indexer-aggregation role — see [[Prowlarr]].
| Whisparr | whisparr.dankserver.local | 6969 | [[ARR-Stack]] |
## Removed Services ## Removed Services
+49 -207
View File
@@ -1,226 +1,68 @@
---
type: reference
title: Jellyfin
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-09-03T00:00:00Z
---
# Jellyfin # Jellyfin
Jellyfin is a free, open-source media server and suite of applications for organizing, managing, and streaming personal media collections (movies, TV shows, music, photos) to networked devices. It is a community fork of Emby, written in C# (.NET), and uses FFmpeg for transcoding. Jellyfin is the free media server on Dankserver — a podman quadlet service
(`jellyfin.service`, generated from `dankserver/jellyfin/quadlet/jellyfin.container`)
serving music, movies, and shows over LAN. It replaced the earlier research
brief; it runs alongside [[Plex]], which remains the primary server for some
clients.
**Host:** Dankserver — **Hostname:** `jellyfin.dankserver.local` — **Docs:** https://jellyfin.org **Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`,
public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
[[Authentik]] outpost (post-auth route proxies `localhost:8096`).
## Facts ## Facts (verified from the live unit, 2026-09-03)
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Latest version | Check https://jellyfin.org/downloads | | Unit | `~/.config/containers/systemd/jellyfin.container` → `jellyfin.service` (user manager), `Restart=always`, `RestartSec=5` |
| Image | `jellyfin/jellyfin:latest` (Docker Hub) or `ghcr.io/jellyfin/jellyfin` | | Image | `ghcr.io/jellyfin/jellyfin:latest` |
| Base | Debian-based | | Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
| Primary port | 8096 (HTTP, web UI + API) | | Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
| HTTPS port | 8920 (optional) | | Media | `Volume=/mnt/entropy:/data:ro` — the whole [[Entropy Volume]] mounted read-only; nothing automated writes to it |
| Client discovery | 7359/udp | | GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
| DLNA | 1900/udp (optional) | | Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
| GPU passthrough | `/dev/dri` (Intel QSV / AMD VA-API) or NVIDIA NVENC | | Health | `curl -f http://localhost:8096/health` every 30 s |
| License | MIT |
## Architecture ## Data layout
```mermaid ```
graph LR /home/dank/Containers/jellyfin/config -> /config (JELLYFIN_DATA_DIR)
U[User] --> C[Caddy :80/:443] /home/dank/Containers/jellyfin/cache -> /cache (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
C --> J[Jellyfin :8096] /mnt/entropy -> /data (read-only media volume)
J --> Lib[/media/movies]
J --> Lib2[/media/shows]
J --> Lib3[/media/music]
J --> Conf[Jellyfin config]
J --> Cache[Jellyfin cache]
``` ```
## Docker Compose (reference — retired 2026-09-01; live mounts use `/home/dank/Containers/jellyfin`, see Deployment below) Library folders point at `/data` paths; because the mount uses the same
absolute path as on the host, library paths survive container recreation.
```yaml ## Ports & access
services:
jellyfin:
image: jellyfin/jellyfin:latest
container_name: jellyfin
restart: unless-stopped
user: "1000:1000"
ports:
- "8096:8096/tcp"
- "8920:8920/tcp"
- "7359:7359/udp"
- "1900:1900/udp"
environment:
- JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
- DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
volumes:
- /opt/jellyfin/config:/config
- /opt/jellyfin/cache:/cache
- /mnt/media/movies:/data/movies:ro
- /mnt/media/shows:/data/shows:ro
devices:
- /dev/dri:/dev/dri
group_add:
- "992" # Host render group GID — verify with `getent group render`
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
interval: 30s
timeout: 10s
retries: 3
```
## Caddy Configuration | Port | Proto | Purpose |
|------|-------|---------|
| 8096 | TCP | Web UI + API (`JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096`) |
| 8920 | TCP | DLNA/UPnP root device control |
| 7359 | UDP | service discovery (client auto-discovery) |
| 1900 | UDP | SSDP discovery |
```caddy Client access paths and the public-exposure policy (auth wall required before
jellyfin.dankserver.local { any public vhost; Plex-style claim tokens do not exist here — Jellyfin uses its
reverse_proxy 127.0.0.1:8096 own users/API keys) follow the fleet [[Authentik]]/[[Anubis]] chain pattern.
transport http { ## Configuration workflow
versions h2c 1.1
}
}
```
Set **Base URL** to `/` (or the path you configure) in Jellyfin Dashboard → Networking. Add `127.0.0.1` to **Known Proxies**. 1. Create libraries pointing at `/data` folders (movies, shows, audio).
2. Set metadata/language preferences (`LANG=en_US.UTF-8` pinned in the unit).
## Storage 3. Configure clients to use `http://192.168.0.235:8096` or the guarded vhost.
| Mount | Purpose | Size Estimate |
|-------|---------|---------------|
| `/home/dank/Containers/jellyfin/config` | SQLite database, settings, cached images | 10–100 GB |
| `/home/dank/Containers/jellyfin/cache` | Transcoding segments, extracted posters | Self-cleaning per session |
| `/mnt/entropy/Movies`, `/mnt/entropy/TVShows` | Media, **read-only binds** (`:ro`) — no automated write path | — |
The database must be on local storage (SSD/NVMe preferred). Media libraries can be on NFS/mergerfs. If media is on NFS, set `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`.
## Deployment (Quadlet)
Since 2026-09-01: `dankserver/jellyfin/quadlet/jellyfin.container` →
`jellyfin.service` (generator-enabled), deployed with
`./scripts/deploy.sh dankserver --service jellyfin`. GPU passthrough uses
`AddDevice=/dev/dri` — plain `Device=` is **not** a quadlet key and makes the
v1 generator reject the whole unit (learned live: "unsupported key 'Device'
in group 'Container'"). Entropy mounts stay `:ro`; the container runs as
root inside the user namespace, so the render-node ACLs apply via the device
cgroup regardless of `GroupAdd=keep-groups`.
## Authentication
- **Built-in auth:** per-user libraries, parental controls, max bitrate
limits. The local `dank` admin account is built-in; its password is
Frank's bootstrap value — never recorded in this repo. (2026-09-01: a
corrupted session-archive reading briefly suggested the password had
changed; verified offline against the stored PBKDF2-SHA512 hash, the
bootstrap value still authenticates.)
- **LDAP via Authentik (live 2026-09-01):** the LDAP-Auth plugin
authenticates users against the authentik LDAP outpost on Dankclaw — see
[[Authentik]] for the outpost unit, provider, and RBAC objects.
### LDAP-Auth plugin (v23.0.0.0, live 2026-09-01)
- **Install from the GitHub releases zip** — `repo.jellyfin.org`'s stable
manifest path 404s here, so fetch
`jellyfin/jellyfin-plugin-ldapauth` release asset
`ldap-authentication_23.0.0.0.zip` from GitHub and unzip into
`config/plugins/LDAP-Auth_23.0.0.0/`. The config lives at
`config/plugins/configurations/LDAP-Auth.xml` — write it directly, then
restart the service.
- **Live config:** server `192.168.0.143:389`, base DN
`dc=franksdank,dc=cloud`, bind DN
`cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`, bind password =
the authentik outpost token (secret — the XML lives on the host only,
never in the repo), search filter
`(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`, admin
filter on `cn=jellyfin_admins,ou=groups,…`, `CreateUsersFromLdap=true`
(auto-create on first login).
- **Attribute separator is a comma.** `LdapSearchAttributes` is
`Split(',')` in the plugin source — a semicolon-separated list
(`uid;cn;mail;displayName`) parses as ONE bogus attribute, the search
silently matches nothing, and the log says `Found no users matching X`.
Use `uid,cn,mail,displayName`.
- **Username attribute is `cn`.** Authentik 2026.x maps `uid` to an opaque
per-user hash, not the username, so `uid=<name>` never matches. Set
`LdapUsernameAttribute=cn` (authentik exposes the username as `cn`).
- **Verify with a throwaway user:** create a test member of
`jellyfin_users` in authentik, then
`POST /Users/AuthenticateByName` with headers `Content-Type:
application/json` **and** `X-Emby-Authorization: MediaBrowser
Client="…", Device="…", DeviceId="…", Version="10.11.0"` — both are
required in 10.11 (missing `X-Emby-Authorization` → 400). 200 + non-admin
policy = LDAP path works. Delete the test user from authentik **and**
Jellyfin afterward.
- **Plugin load lines only show in `journalctl --user -u jellyfin.service`**
(`Loaded plugin: LDAP-Auth 23.0.0.0`) — `podman logs` does not carry the
loader lines in this image.
## Metadata Scraping
| Provider | Content | Type |
|----------|---------|------|
| TheMovieDb (TMDb) | Movies, TV shows, posters, fan art, overviews | Default |
| OMDb API | Movies only, English metadata | Default |
| TheTVDB | TV shows, episodes | Plugin |
| fanart.tv | Extra artwork, banners, thumbnails | Plugin |
| AniDB | Anime metadata | Plugin |
| MusicBrainz | Music metadata | Plugin |
| Open Subtitles | Subtitle downloads | Plugin |
| Intro Skipper | TV show intro detection | Plugin |
## Clients
| Platform | App | Quality |
|----------|-----|---------|
| Web | Jellyfin Web | Excellent |
| Android | Jellyfin (Google Play / F-Droid) | Excellent |
| Android TV / Fire TV | Jellyfin | Good |
| iOS / iPadOS | Swiftfin (third-party) | Good |
| Desktop | Jellyfin Media Player (mpv-based) | Excellent |
| Roku | Jellyfin (Channel Store) | Good |
| Kodi | Jellyfin addon | Excellent |
| Samsung/LG Smart TVs | TV browser | Fair |
## Jellyfin vs Plex
| Feature | Jellyfin | Plex |
|---------|----------|------|
| Cost | Free, no tiers | Free core; Plex Pass ($5/mo or $120 lifetime) |
| License | MIT | Proprietary |
| Account required | No | Yes |
| Hardware transcoding | Free | Plex Pass required |
| Mobile sync | Free | Plex Pass |
| Live TV / DVR | Free | Plex Pass |
| Client apps | Good, improving | Excellent, widest device support |
Run Jellyfin alongside Plex for an incremental migration. Point Jellyfin at the same media folders as Plex. Watch history and metadata will not transfer cleanly.
## Resource Requirements
- **Idle / Direct Play:** ~200–500 MB RAM, <5% CPU. Negligible.
- **1080p transcode (CPU):** 40–60% CPU on a modern multi-core CPU.
- **1080p transcode (with QSV):** 3–5% CPU.
- **4K HDR transcode (with QSV):** 5–15% CPU.
- **Minimum host RAM:** 4 GB. 8 GB comfortable for transcoding.
## Deployment Checklist
1. Verify GPU on host: `ls -l /dev/dri/`, `lspci | grep -i vga`, `getent group render`.
2. Install media drivers: `sudo apt install intel-media-va-driver-non-free vainfo intel-gpu-tools`.
3. Create directories: `/home/dank/Containers/jellyfin/config`, `/home/dank/Containers/jellyfin/cache`.
4. Write compose file, verify device and GID.
5. Deploy: `podman compose up -d`.
6. Test GPU passthrough: `podman exec jellyfin /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128`.
7. Configure Caddy reverse proxy. Set Known Proxies in Jellyfin.
8. Add media libraries in Dashboard.
9. Install plugins: Open Subtitles, fanart.tv, Intro Skipper.
10. Test playback on Android/Chrome.
## Gotchas
- **GPU passthrough silently failing:** `/dev/dri` must be passed and the correct render node used. Verify with `vainfo` inside the container.
- **Wrong render group GID:** Common GID is 992, but varies per host. Run `getent group render` to find it.
- **Subtitle burn-in triggers full transcode:** PGS (image-based) subtitles force full video transcoding. SRT/ASS (text-based) play fine.
- **Database on NFS:** SQLite on network storage causes slow startup and corruption risk. Keep `/config` on local storage.
- **Client bitrate limit too low:** Many clients default to 20 Mbps. Set Max Streaming Bitrate to Original or Maximum.
- **Naming convention mismatches:** Non-standard folder/file names cause missing metadata. Use `Movie Title (Year)` for movies and `Show Name/Season 01/Show Name S01E01.mkv` for TV.
## See Also ## See Also
- [[Dankserver Stack]] - [[Plex]] — co-resident media server on the same volume
- [[ARR-Stack]] — automated download into Jellyfin libraries - [[Entropy Volume]] — the 91 T array mounted read-only at `/data`
- [[Copyparty]] — alternative file viewer for existing collections - [[Caddy]] · [[Anubis]] · [[Authentik]] — edge chain for `jelly.franksdank.cloud`
- Plex — existing media server on this host
+6 -6
View File
@@ -1,10 +1,10 @@
# Plex # Plex
Plex Media Server is the **containerized media server on Dankserver**: a Plex Media Server is a **containerized media server on Dankserver**: a podman
podman quadlet (`plex.service`) under the systemd **user** generator, quadlet (`plex.service`) under the systemd **user** generator, generated from
generated from `dankserver/plex/quadlet/plex.container`. It is the only media `dankserver/plex/quadlet/plex.container`. It shares the host with a second live
server actually listening on this host. Jellyfin, by contrast, is a research media server, [[Jellyfin]] (`jellyfin.service`, listening on 8096); both serve
brief on this host, not a running service (see [[Jellyfin]]). from the same read-only Entropy Volume mount.
**Host:** Dankserver — **Ports:** 32400/tcp (PMS web + API), 32400 is the **Host:** Dankserver — **Ports:** 32400/tcp (PMS web + API), 32400 is the
only port PMS needs for client streaming; LAN clients connect directly, only port PMS needs for client streaming; LAN clients connect directly,
@@ -21,7 +21,7 @@ public traffic can be fronted by Caddy like every other vhost.
| Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover | | Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
| Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration | | Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
| Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` | | Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
| GPU | none by decision — CPU decode only; the RTX 3090 stays dedicated to `frank-default-small` | | GPU | none by decision — CPU decode only; the RTX 3090 stays dedicated to `frank-small` |
| Deploy | `./scripts/deploy.sh dankserver --service plex` | | Deploy | `./scripts/deploy.sh dankserver --service plex` |
| Docs | https://support.plex.tv | | Docs | https://support.plex.tv |
+15 -12
View File
@@ -2,14 +2,15 @@
RomM (ROM Manager) is a self-hosted ROM library manager and player. It scans a ROM collection, enriches it with metadata and cover art from multiple providers, and serves it through a clean web interface. RomM also supports browser-based emulation via EmulatorJS and native emulator streaming. RomM (ROM Manager) is a self-hosted ROM library manager and player. It scans a ROM collection, enriches it with metadata and cover art from multiple providers, and serves it through a clean web interface. RomM also supports browser-based emulation via EmulatorJS and native emulator streaming.
**Host:** Dankserver — **Hostname:** `romm.dankserver.local` — **Docs:** https://docs.romm.app **Host:** Dankserver — **Access:** `romm.franksdank.cloud` guarded by the
Anubis → Authentik chain ([[Caddy]] routes it post-auth to
`192.168.0.235:8085`) — **Docs:** https://docs.romm.app
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Latest version | 5.2.0 (released 2026-07-29) | | Running image | `ghcr.io/rommapp/romm:latest` (rolling tag — no pin; the version claims on this page were true at research time; verify against the live container before relying on them) |
| Image | `ghcr.io/rommapp/romm:latest` |
| Port | 8085 (host) → 8080 (container) | | Port | 8085 (host) → 8080 (container) |
| Database | MariaDB (default); PostgreSQL optional | | Database | MariaDB (default); PostgreSQL optional |
| Cache / Queue | Redis or Valkey (required for task queue) | | Cache / Queue | Redis or Valkey (required for task queue) |
@@ -21,7 +22,9 @@ RomM (ROM Manager) is a self-hosted ROM library manager and player. It scans a R
## Architecture ## Architecture
RomM 5.2.0 requires three supporting services. The compose stack includes `romm`, `mariadb`, and `redis` (or `valkey`). Current RomM releases require MariaDB plus Redis (or Valkey) for the task
queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit),
not valkey.
```mermaid ```mermaid
graph LR graph LR
@@ -68,7 +71,7 @@ These are set via environment variables or in `config.yml`:
| `MOBYGAMES_API_KEY` | MobyGames | Paid hobbyist API. Note: RomM recommends Screenscraper as free alternative. | | `MOBYGAMES_API_KEY` | MobyGames | Paid hobbyist API. Note: RomM recommends Screenscraper as free alternative. |
| `RETROACHIEVEMENTS_API_KEY` | RetroAchievements | Hash-based achievements via bundled RAHasher binary. | | `RETROACHIEVEMENTS_API_KEY` | RetroAchievements | Hash-based achievements via bundled RAHasher binary. |
| `STEAMGRIDDB_API_KEY` | SteamGridDB | Custom cover art via the search cover button. | | `STEAMGRIDDB_API_KEY` | SteamGridDB | Custom cover art via the search cover button. |
| `HASHEOUS_API_ENABLED` | Hasheous | Set to `true`. Free, hash-based, proxies IGDB data. Fast scans. | | `HASHEOUS_API_ENABLED` | Hasheous | Set to `true` in the live unit env file. Free, hash-based, proxies IGDB data. Fast scans. |
| `PLAYMATCH_API_ENABLED` | Playmatch | Set to `true`. Multi-provider hash matching (IGDB, SS, SteamGridDB, RA). | | `PLAYMATCH_API_ENABLED` | Playmatch | Set to `true`. Multi-provider hash matching (IGDB, SS, SteamGridDB, RA). |
### Config File ### Config File
@@ -117,15 +120,15 @@ Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
## Volumes ## Volumes
The live unit mounts one root bind — `Volume=/home/dank/Containers/romm:/romm`
— so `/romm/config`, `/romm/library`, `/romm/resources`, and `/romm/assets`
all resolve inside a single host-side data root:
| Container path | Host path | | Container path | Host path |
|----------------|-----------| |----------------|-----------|
| `/romm/library` | `/home/dank/Containers/romm/library` | | `/romm/*` (single root bind) | `/home/dank/Containers/romm/*` |
| `/romm/resources` | `/home/dank/Containers/romm/resources` | | `/var/lib/mysql` (romm-mariadb) | `/home/dank/Containers/romm/mariadb` (mapped subuid owner — never chown) |
| `/romm/config` | `/home/dank/Containers/romm/config` | | `/data` (romm-redis) | `/home/dank/Containers/romm/redis` (mapped subuid owner) |
| `/var/lib/mysql` | `/home/dank/Containers/romm/mariadb` (mapped subuid owner — never chown) |
| `/data` (redis) | `/home/dank/Containers/romm/redis` (mapped subuid owner) |
The library volume contains ROM files and subdirectories per platform. The resources volume stores cached artwork, metadata, and the asset cache. The config volume holds `config.yml` and other configuration.
The cutover rescued RomM's anonymous `/romm` volume (assets, plus empty The cutover rescued RomM's anonymous `/romm` volume (assets, plus empty
mountpoint stubs that shadowed nothing) into the data root before first unit mountpoint stubs that shadowed nothing) into the data root before first unit
+39 -23
View File
@@ -1,8 +1,19 @@
---
type: reference
title: Lidarr
description: Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
tags: [lidarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Lidarr # Lidarr
Lidarr manages music albums in the *arr stack on Dankserver. It automatically searches, downloads, and organizes albums from indexers into the Jellyfin music library. Lidarr manages music albums in the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
not on Dankserver. It searches, downloads, and organizes albums from indexers
into the Jellyfin music library.
**Host:** Dankserver — **Hostname:** `lidarr.dankserver.local` — **Docs:** https://wiki.servarr.com/lidarr **Host:** FrankBook-Pro (podman machine) — **Access:** direct port `8686` (no reverse-proxy vhost; the *arr stack is exempt by construction from the Anubis guard chain) — **Docs:** https://wiki.servarr.com/lidarr
## Facts ## Facts
@@ -10,36 +21,41 @@ Lidarr manages music albums in the *arr stack on Dankserver. It automatically se
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/lidarr:latest` | | Image | `lscr.io/linuxserver/lidarr:latest` |
| Port | 8686 (host) → 8686 (container) | | Port | 8686 (host) → 8686 (container) |
| Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Config root | `/Volumes/COMPLETE/lidarr-data:/config` |
| Min RAM | 256 MB | | Min RAM | 256 MB |
| Metadata | MusicBrainz | | Metadata | MusicBrainz |
## Docker Compose ## Deployment (current)
```yaml Quadlet unit `lidarr.container` from this repo (`frankbook-pro/lidarr/quadlet/`)
lidarr: is installed inside the podman machine at
image: lscr.io/linuxserver/lidarr:latest `/home/core/.config/containers/systemd/lidarr.service` by the frankbook-pro
container_name: lidarr handler in `scripts/deploy-remote.sh`. Unit shape:
restart: unless-stopped
environment: ```ini
- PUID=1000 [Container]
- PGID=1000 ContainerName=lidarr
- TZ=America/New_York Image=lscr.io/linuxserver/lidarr:latest
- UMASK_SET=022 Environment=PUID=0
volumes: Environment=PGID=0
- /opt/lidarr/config:/config Environment=TZ=America/Los_Angeles
- /mnt/media/music:/music Environment=UMASK_SET=022
- /mnt/media/downloads:/downloads Volume=/Volumes/COMPLETE/lidarr-data:/config
ports: PublishPort=8686:8686
- "8686:8686"
networks: [appnet] [Service]
TimeoutStartSec=900
Restart=always
``` ```
## Configuration ## Configuration
1. Add Prowlarr as the indexer source (prefer torrent-based indexers for FLAC). 1. Add Prowlarr as the indexer source (prefer torrent-based indexers for FLAC —
2. Add qBittorrent as the download client with the `music` category. the Nyaa feed carries music categories; see [[Prowlarr]]).
2. Add qBittorrent as the download client with the `music` category ([[qBittorrent]]).
3. Set quality profile (Lossless → High Bitrate → Standard). 3. Set quality profile (Lossless → High Bitrate → Standard).
4. Set root folder to `/music`. 4. Set the root folder to the music root under the COMPLETE volume.
5. Add artists and verify downloads. 5. Add artists and verify downloads.
## Gotchas ## Gotchas
+62 -27
View File
@@ -1,47 +1,82 @@
---
type: reference
title: MariaDB (RomM Backend)
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
tags: [mariadb, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
# MariaDB (RomM Backend) # MariaDB (RomM Backend)
MariaDB 11 provides the database backend for RomM on Dankserver. RomM 5.2.0 requires MariaDB (or PostgreSQL) for its data store. MariaDB 11 provides the database backend for [[RomM]] on Dankserver. RomM's
current release requires MariaDB (or PostgreSQL) for its data store; this
deployment uses MariaDB.
**Host:** Dankserver — **Docs:** https://mariadb.com/kb/en/docker-mariadb-information/ **Host:** Dankserver — container name `romm-mariadb`, unit
`romm-mariadb.service` (user-manager quadlet from
`dankserver/romm/quadlet/romm-mariadb.container`).
## Facts ## Facts (verified from the live unit)
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `docker.io/mariadb:11` | | Image | `docker.io/library/mariadb:11` |
| Port | 3306 (internal, within `appnet`) | | Port | 3306 — internal only; the unit publishes **no** port, so the DB is unreachable from the host network by construction |
| Database name | `romm` | | Network | `dankserver_appnet` with `alias=mariadb` — RomM reaches it via `DB_HOST=mariadb` |
| Data volume | `/opt/romm/mariadb` | | Database / user | `romm` / `romm` |
| Data volume | `/home/dank/Containers/romm/mariadb:/var/lib/mysql` |
| Secrets | supplied via `EnvironmentFile=/home/dank/Containers/romm/env` (mode 0600, host-local — `MARIADB_ROOT_PASSWORD` and friends never enter the repo) |
## Docker Compose ## Unit shape (as deployed)
```yaml ```ini
mariadb: [Unit]
image: docker.io/mariadb:11 Description=RomM mariadb container
container_name: romm-mariadb After=network.target
restart: unless-stopped Wants=romm-redis.service
environment:
MARIADB_ROOT_PASSWORD: "${MARIADB_ROOT_PASSWORD}" [Container]
MARIADB_DATABASE: romm ContainerName=romm-mariadb
MARIADB_USER: "${ROMM_DB_USER}" Image=docker.io/library/mariadb:11
MARIADB_PASSWORD: "${ROMM_DB_PASSWD}" Network=dankserver_appnet:alias=mariadb
volumes: EnvironmentFile=/home/dank/Containers/romm/env
- /opt/romm/mariadb:/var/lib/mysql Environment=MARIADB_DATABASE=romm
healthcheck: Environment=MARIADB_USER=romm
test: ["CMD", "mariadb-admin", "ping", "-h", "localhost"] Volume=/home/dank/Containers/romm/mariadb:/var/lib/mysql
interval: 10s HealthCmd=mariadb-admin ping -h localhost
timeout: 5s HealthInterval=30s
retries: 3 HealthTimeout=10s
HealthRetries=3
[Service]
Restart=always
RestartSec=5
``` ```
`MARIADB_ROOT_PASSWORD` and `MARIADB_PASSWORD` are read from the env file at
container-create time (`EnvironmentFile=` appears in both `[Container]` and
`[Service]` so `podman generate systemd`-style regeneration keeps them). The
root password is a real secret — per repo secret rules it lives only in the
host-local env file, never in git.
## Boot order
RomM declares `Wants=romm-mariadb.service` + `After=romm-mariadb.service`, so
a cold boot starts the database first; the health ping (`mariadb-admin ping`)
is what marks the unit healthy for anything gating on it.
## Backup ## Backup
Back up the volume directory periodically: Back up the volume directory periodically (stop-free: rsync of a quiet DB dir):
```bash ```bash
sudo rsync -a /opt/romm/mariadb /mnt/backup/romm-mariadb/ sudo rsync -a /home/dank/Containers/romm/mariadb /mnt/backup/romm-mariadb/
``` ```
For consistency-critical dumps prefer `mariadb-dump` over file copy; the data
dir copy is only safe while RomM traffic is idle.
## See Also ## See Also
- [[RomM]] — service that uses this database - [[RomM]] — service that uses this database
- [[redis]] — the sibling cache/queue container on the same network
+54 -39
View File
@@ -1,20 +1,55 @@
---
type: reference
title: Prowlarr
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
tags: [prowlarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Prowlarr # Prowlarr
Prowlarr is the indexer management layer for the *arr stack. It hosts the Prowlarr is the indexer management layer for the *arr stack. It hosts the
indexer definitions and exposes them as Torznab feeds and downloads; indexer definitions and exposes them as Torznab feeds and downloads;
Sonarr, Radarr, Lidarr, and Readarr query those feeds and receive results. Sonarr, Radarr, Lidarr, and Readarr query those feeds and receive results.
**Host:** Dankserver — **Hostname:** `prowlarr.dankserver.local` — **Docs:** https://wiki.servarr.com/prowlarr **Host:** FrankBook-Pro (podman machine) — **Access:** direct port `9696` (no
reverse-proxy vhost; the *arr stack is exempt by construction from the Anubis
guard chain) — **Docs:** https://wiki.servarr.com/prowlarr
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/prowlarr:latest` | | Image | `lscr.io/linuxserver/prowlarr:latest` (runs v2.5.2 line) |
| Port | 9696 (host) → 9696 (container) | | Port | 9696 (host) → 9696 (container) |
| Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Config root | `/Volumes/COMPLETE/prowlarr-data:/config` |
| Min RAM | 256 MB | | Min RAM | 256 MB |
| Auth | Forms (web UI) + API key (`/api/v1/...` and feed routes) | | Auth | Forms (web UI) + API key (`/api/v1/...` and feed routes) |
## Deployment (current)
Quadlet unit `prowlarr.container` from this repo
(`frankbook-pro/prowlarr/quadlet/`), installed inside the podman machine at
`/home/core/.config/containers/systemd/prowlarr.service` by the frankbook-pro
handler in `scripts/deploy-remote.sh`:
```ini
[Container]
ContainerName=prowlarr
Image=lscr.io/linuxserver/prowlarr:latest
Environment=PUID=0
Environment=PGID=0
Environment=TZ=America/Los_Angeles
Environment=UMASK_SET=022
Volume=/Volumes/COMPLETE/prowlarr-data:/config
PublishPort=9696:9696
[Service]
TimeoutStartSec=900
Restart=always
```
## Live wiring (verified end-to-end 2026-08-30) ## Live wiring (verified end-to-end 2026-08-30)
Indexer queries flow **through** Prowlarr; a grab hands off to the querying Indexer queries flow **through** Prowlarr; a grab hands off to the querying
@@ -35,35 +70,17 @@ http://prowlarr:9696/{indexerId}/download?link=...&file=... (torrent handoff)
The global API key is in `config.xml` (`<ApiKey>`); per-application keys are The global API key is in `config.xml` (`<ApiKey>`); per-application keys are
only used by Prowlarr's own sync calls, NOT by the feeds. The arr download only used by Prowlarr's own sync calls, NOT by the feeds. The arr download
clients authenticate to qBittorrent with the permanent WebUI password stored clients authenticate to qBittorrent with the permanent WebUI password stored
in `/home/dank/.qbpass` (mode 600) on Dankserver. in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver — see
[[qBittorrent]] for why that password must be pinned in the conf file.
**Readarr caveat:** Readarr is abandoned upstream (last release 0.4.x); its **Readarr caveat:** Readarr is abandoned upstream; its qBittorrent client
qBittorrent client cannot authenticate against qB 5 (`Authentication cannot authenticate against newer qB Web-API versions (`Authentication
failure` on the connection test while Sonarr/Lidarr/Radarr pass with the failure` on the connection test while Sonarr/Lidarr/Radarr pass with the same
same credentials). The books pipeline is parked pending a Readarr successor; credentials — root cause: qB ≥ 5's 204 empty-body login success, fixed fleet-
the indexer row itself is healthy. wide by pinning qB to 4.6.7). The books pipeline is parked pending a Readarr
successor; the indexer row itself is healthy.
## Docker Compose ## Gotchas (verified live 2026-08-30)
```yaml
prowlarr:
image: lscr.io/linuxserver/prowlarr:latest
container_name: prowlarr
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
- UMASK_SET=022
volumes:
- /opt/prowlarr/config:/config
- /mnt/media/downloads:/downloads:ro
ports:
- "9696:9696"
networks: [appnet]
```
## Gotchas (all verified live 2026-08-30)
- **The feed route is `/{indexerId}/api`.** There is no `/torznab/{id}/feed` - **The feed route is `/{indexerId}/api`.** There is no `/torznab/{id}/feed`
— that path (and any other unmatched path) hits the Forms-auth catch-all — that path (and any other unmatched path) hits the Forms-auth catch-all
@@ -92,12 +109,13 @@ prowlarr:
- Instance name/`InstanceName=Prowlarr` in `config.xml` does NOT prefix the - Instance name/`InstanceName=Prowlarr` in `config.xml` does NOT prefix the
API/feed routes on this build (they answer at root; the `/Prowlarr/...` API/feed routes on this build (they answer at root; the `/Prowlarr/...`
prefix 302s to the SPA). prefix 302s to the SPA).
- **Application auto-sync cannot manage these rows** (they carry - **Application auto-sync model (v2):** Applications are first-class rows in
`SyncLevel=2`): `POST /api/v1/applications/test` fails with the `Applications` table; each of the four app rows (sonarr/radarr/lidarr/
`Unable to cast object of type 'NzbDrone.Core.Thing…'` — Prowlarr v1 readarr) carries the real baseUrl+apiKey with `SyncLevel=FullSync`. The
cannot deserialize Radarr 6.3's newer response types. The manual Torznab earlier v1-era failure where `POST /api/v1/applications/test` threw
rows above are therefore the real integration; sync neither duplicates `Unable to cast object of type 'NzbDrone.Core.Thing…'` (v1 could not
nor clobbers them. deserialize Radarr 6.3's newer response types) is historical — it was the
reason the manual Torznab rows were created in the first place.
- Prowlarr runs `AuthenticationMethod=Forms` + `AuthenticationRequired=Enabled` - Prowlarr runs `AuthenticationMethod=Forms` + `AuthenticationRequired=Enabled`
healthily — the DryIoc crash combo was `None`+`Enabled` only (see healthily — the DryIoc crash combo was `None`+`Enabled` only (see
[[radarr]] auth notes). [[radarr]] auth notes).
@@ -105,7 +123,4 @@ prowlarr:
## See Also ## See Also
- [[ARR-Stack]] — full *arr stack documentation - [[ARR-Stack]] — full *arr stack documentation
- [[radarr]] - [[Radarr]] · [[Sonarr]] · [[Lidarr]] · [[Readarr]] · [[qBittorrent]]
- [[sonarr]]
- [[lidarr]]
- [[readarr]]
+66 -38
View File
@@ -1,27 +1,44 @@
---
type: reference
title: qBittorrent
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
timestamp: 2026-09-03T00:00:00Z
---
# qBittorrent # qBittorrent
qBittorrent is the torrent download client for the *arr stack on Dankserver. It receives download requests from Sonarr, Radarr, Lidarr, and Readarr via category tagging. qBittorrent is the torrent download client for the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
not on Dankserver. It receives download requests from Sonarr, Radarr, Lidarr,
and Readarr via category tagging.
**Host:** Dankserver — **Hostname:** `qbittorrent.dankserver.local` — **Docs:** https://www.qbittorrent.org **Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`,
published by the [[Gluetun]] unit because qBittorrent shares that container's
network namespace (a netns-sharer publishes no ports of its own).
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/qbittorrent:latest` | | Image | `lscr.io/linuxserver/qbittorrent:4.6.7` — **pinned**, see the pin rationale below |
| Port | 8080 (host) → 8080 (container) | | Network | `Network=gluetun.container` (torrent egress exits via the WireGuard/ProtonVPN tunnel) |
| Min RAM | 512 MB | | Ports | 8080 (WebUI) and 6881 TCP+UDP published on the gluetun unit |
| Auth | Web UI username/password; LAN mode available | | Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Volumes | `/Volumes/COMPLETE/qbittorrent-data:/config`, plus `/Volumes/COMPLETE` and `/Volumes/INCOMPLETE` mounted at identical paths |
| Auth | Web UI username/password (`admin` + generated password); LAN bypass not used |
## Configuration ## Why the 4.6.7 pin exists (2026-09-01)
1. Keep Web UI authentication ON (LAN bypass not needed — the arr apps Readarr's abandoned qBittorrent client rejects qB ≥ 5's **204 empty-body login
authenticate with the API) and give them a permanent password. success** as `Authentication failure` — a runtime break, not just an API-create
2. Create download categories per app (`radarr`, `sonarr`, `lidarr`, issue. The fleet pins `lscr.io/linuxserver/qbittorrent:4.6.7` while Readarr is
`readarr`): the arr download clients route category-per-app, and Prowlarr in the stack. **Never unpin to `latest` while Readarr is in the fleet.**
feeds hand grabs to each app's own client.
3. Set max active downloads: Settings → Downloads (recommended: 5 global, Second trap on any version flip: qB 5.x's shutdown conf rewrite leaves a
3 per category). `WebUI\Password_PBKDF2=@ByteArray(...)` line that 4.6.7 rejects — every login
then returns "invalid credentials" (the arrs included). Fix = rewrite the same
conf line with the same password while qB is stopped.
## Persistent WebUI credentials (verified 2026-08-30) ## Persistent WebUI credentials (verified 2026-08-30)
@@ -46,31 +63,34 @@ val = (base64.b64encode(salt) + b":" + base64.b64encode(h)).decode()
Then `podman start qbittorrent` and confirm login survives a Then `podman start qbittorrent` and confirm login survives a
`podman restart` — the temp-password rotation stops the moment stored `podman restart` — the temp-password rotation stops the moment stored
credentials verify. The chosen value lives host-local in credentials verify. The chosen value lives host-local in `/home/dank/.qbpass`
`/home/dank/.qbpass` (mode 600, owner `dank`) — never in git; every arr (mode 600, owner `dank`) — never in git; every arr download client and
download client and `prowlarr.md` reference it. (The classic [[Prowlarr]] reference it. (The classic `@ByteArray(salt+hash)` form is WRONG
`@ByteArray(salt+hash)` form is WRONG for qb5: login 401s on it; verify() for qb5: login 401s on it; verify() base64-decodes the two halves separately.)
base64-decodes the two halves separately.)
## Docker Compose ## Deployment (current)
```yaml Quadlet unit `qbittorrent.container` from this repo
qbittorrent: (`frankbook-pro/qbittorrent/quadlet/`), installed inside the podman machine at
image: lscr.io/linuxserver/qbittorrent:latest `/home/core/.config/containers/systemd/qbittorrent.service` by the
container_name: qbittorrent frankbook-pro handler in `scripts/deploy-remote.sh`:
restart: unless-stopped
environment: ```ini
- PUID=1000 [Container]
- PGID=1000 ContainerName=qbittorrent
- TZ=America/New_York Image=lscr.io/linuxserver/qbittorrent:4.6.7
- UMASK_SET=022 Network=gluetun.container
volumes: Environment=PUID=0
- /opt/qbittorrent/config:/config Environment=PGID=0
- /mnt/media/downloads:/downloads Environment=TZ=America/Los_Angeles
- /mnt/media/downloads/incomplete:/downloads/incomplete Environment=UMASK_SET=022
ports: Volume=/Volumes/COMPLETE/qbittorrent-data:/config
- "8080:8080" Volume=/Volumes/COMPLETE:/Volumes/COMPLETE
networks: [appnet] Volume=/Volumes/INCOMPLETE:/Volumes/INCOMPLETE
[Service]
TimeoutStartSec=900
Restart=always
``` ```
## Gotchas (verified live) ## Gotchas (verified live)
@@ -81,9 +101,17 @@ qbittorrent:
download queue" in the arr logs) after any restart. download queue" in the arr logs) after any restart.
- API login from `curl` needs a `Referer: http://<host>:<port>/` header - API login from `curl` needs a `Referer: http://<host>:<port>/` header
(CSRF protection) or POSTs get rejected even with a valid SID. (CSRF protection) or POSTs get rejected even with a valid SID.
- Cookie-name quirk by version: qB 5.x names its WebUI session cookie
`QBT_SID_<port>`; 4.6.7 uses plain `SID`. Any automation must parse the
cookie generically (e.g. `grep -oE '[A-Za-z_]*SID[A-Za-z0-9_]*=[^;[:space:]]*'`)
instead of hard-coding a port-suffixed name, or it fails with 403 cascades
that never reach qB as failed logins.
- Egress verification for this stack must run **inside the gluetun container**
(`podman exec gluetun wget -qO- https://ipinfo.io/json`); running the same
probe one ssh-context out returns the residential IP — a false leak alarm.
- Ensure umask is set to `022` so downloaded files are readable by arr apps. - Ensure umask is set to `022` so downloaded files are readable by arr apps.
- Monitor bandwidth limits to avoid saturating the network.
## See Also ## See Also
- [[ARR-Stack]] — full *arr stack documentation - [[ARR-Stack]] — full *arr stack documentation
- [[Gluetun]] — the WireGuard tunnel this container shares
+42 -25
View File
@@ -1,52 +1,69 @@
---
type: reference
title: Radarr
description: Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
tags: [radarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Radarr # Radarr
Radarr manages movies in the *arr stack on Dankserver. It automatically searches, downloads, and organizes movies from indexers into the Jellyfin media library. Radarr manages movies in the *arr stack. The stack runs on **FrankBook-Pro**
(the Mac mini) as quadlet units inside its podman machine — not on Dankserver.
It searches, downloads, and organizes movies from indexers into the Jellyfin
movie library.
**Host:** Dankserver — **Hostname:** `radarr.dankserver.local` — **Docs:** https://wiki.servarr.com/radarr **Host:** FrankBook-Pro (podman machine) — **Access:** direct port `7878` (no reverse-proxy vhost; the *arr stack is exempt by construction from the Anubis guard chain) — **Docs:** https://wiki.servarr.com/radarr
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/radarr:latest` (v6 stable) | | Image | `lscr.io/linuxserver/radarr:latest` (v6 line; v6.3.x verified live 2026-08-30) |
| Port | 7878 (host) → 7878 (container) | | Port | 7878 (host) → 7878 (container) |
| Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Config root | `/Volumes/COMPLETE/radarr-data:/config` |
| Min RAM | 512 MB | | Min RAM | 512 MB |
| Metadata | TMDB / TheMovieDB API | | Metadata | TMDB / TheMovieDB API |
## Docker Compose ## Deployment (current)
```yaml Quadlet unit `radarr.container` from this repo (`frankbook-pro/radarr/quadlet/`)
radarr: is installed inside the podman machine at
image: lscr.io/linuxserver/radarr:latest `/home/core/.config/containers/systemd/radarr.service` by the frankbook-pro
container_name: radarr handler in `scripts/deploy-remote.sh`. Unit shape:
restart: unless-stopped
environment: ```ini
- PUID=1000 [Container]
- PGID=1000 ContainerName=radarr
- TZ=America/New_York Image=lscr.io/linuxserver/radarr:latest
- UMASK_SET=022 Environment=PUID=0
volumes: Environment=PGID=0
- /opt/radarr/config:/config Environment=TZ=America/Los_Angeles
- /mnt/media/movies:/movies Environment=UMASK_SET=022
- /mnt/media/downloads:/downloads Volume=/Volumes/COMPLETE/radarr-data:/config
ports: PublishPort=7878:7878
- "7878:7878"
networks: [appnet] [Service]
TimeoutStartSec=900
Restart=always
``` ```
## Configuration ## Configuration
1. Add Prowlarr as the indexer source. 1. Add Prowlarr as the indexer source (the YTS indexer row feeds Radarr — see [[Prowlarr]]).
2. Add qBittorrent as the download client with the `movies` category. 2. Add qBittorrent as the download client with the `movies` category ([[qBittorrent]]).
3. Set quality profile and cutoff quality. 3. Set quality profile and cutoff quality.
4. Set root folder to `/movies`. 4. Set the root folder to the movies root under the COMPLETE volume (Radarr
imports from where [[qBittorrent]] drops completed downloads; first verified
grab+import was a 2160p Dark Knight test on 2026-08-30).
5. Add movies and verify downloads. 5. Add movies and verify downloads.
## Gotchas ## Gotchas
- Radarr v6 is the current stable (v6.3.x as of 2026-08). `:latest` points to the current release. - Radarr v6 is the current stable (v6.3.x as of 2026-08). `:latest` points to the current release.
- Movie renaming produces `Title (Year)` format. Jellyfin scanner handles this. - Movie renaming produces `Title (Year)` format. Jellyfin scanner handles this.
- File permissions: qBittorrent downloads as PUID; arr apps must read/move files. - File permissions: qBittorrent downloads as PUID=0; arr apps must read/move files.
## See Also ## See Also
+56 -30
View File
@@ -1,55 +1,81 @@
---
type: reference
title: Readarr
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
tags: [readarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Readarr # Readarr
> **RETIRED (May 2026).** The Servarr Team retired Readarr: its metadata became unusable and the community migration to Open Library stalled. There is no upstream development. Alternatives: the Bookshelf community fork, or the Calibre content server. See [Readarr (Retired)](https://wiki.servarr.com/readarr). > **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
> metadata became unusable and the community migration to Open Library stalled.
> There is no upstream development. Alternatives: the Bookshelf community fork,
> or the Calibre content server. This instance still runs on a pinned image
> tag; see below.
Readarr manages ebooks and audiobooks in the *arr stack on Dankserver. It automatically searches, downloads, and organizes books into the Calibre library. Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
not on Dankserver. It organizes books into the Calibre library that
[[Calibre-Web]] serves.
**Host:** Dankserver — **Hostname:** `readarr.dankserver.local` — **Docs:** https://wiki.servarr.com/readarr **Host:** FrankBook-Pro (podman machine) — **Access:** direct port `8787` (no reverse-proxy vhost; the *arr stack is exempt by construction from the Anubis guard chain) — **Docs:** https://wiki.servarr.com/readarr
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/readarr:latest` (v0.2.x development) | | Image | `lscr.io/linuxserver/readarr:amd64-1.0.980` — **pinned tag** (LSIO ships no untagged `latest` for readarr; the tag was pinned after a manifest probe) |
| Port | 8787 (host) → 8787 (container) | | Port | 8787 (host) → 8787 (container) |
| Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Config root | `/Volumes/COMPLETE/readarr-data:/config` |
| Min RAM | 256 MB | | Min RAM | 256 MB |
| Metadata | Goodreads / StoryGraph | | Metadata | Goodreads / StoryGraph (provider status at retirement) |
## Docker Compose ## Deployment (current)
```yaml Quadlet unit `readarr.container` from this repo (`frankbook-pro/readarr/quadlet/`)
readarr: is installed inside the podman machine at
image: lscr.io/linuxserver/readarr:latest `/home/core/.config/containers/systemd/readarr.service` by the frankbook-pro
container_name: readarr handler in `scripts/deploy-remote.sh`. Unit shape:
restart: unless-stopped
environment: ```ini
- PUID=1000 [Container]
- PGID=1000 ContainerName=readarr
- TZ=America/New_York Image=lscr.io/linuxserver/readarr:amd64-1.0.980
- UMASK_SET=022 Environment=PUID=0
volumes: Environment=PGID=0
- /opt/readarr/config:/config Environment=TZ=America/Los_Angeles
- /mnt/media/books:/books Environment=UMASK_SET=022
- /mnt/media/downloads:/downloads Volume=/Volumes/COMPLETE/readarr-data:/config
ports: PublishPort=8787:8787
- "8787:8787"
networks: [appnet] [Service]
TimeoutStartSec=900
Restart=always
``` ```
## Configuration ## Configuration
1. Add Prowlarr as the indexer source (torrent indexers with book content). 1. Add Prowlarr as the indexer source (torrent indexers with book content;
2. Add qBittorrent as the download client with the `books` category. the Byrutor row feeds Readarr — see [[Prowlarr]]).
2. The qBittorrent download client is **parked**: Readarr's abandoned client
fails auth against newer qB Web-API versions (see [[qBittorrent]] for the
4.6.7 pin rationale); re-enable only after the image changes. Until then,
books are imported manually or via Calibre tooling.
3. Set author profile (format preference: epub, mobi, etc.). 3. Set author profile (format preference: epub, mobi, etc.).
4. Set quality profile for audiobooks (320 kbps MP3, 128 kbps, lossless). 4. Set quality profile for audiobooks (320 kbps MP3, 128 kbps, lossless).
5. Set root folder to `/books`. 5. Set the root folder to the books root under the COMPLETE volume.
6. Add authors and verify downloads.
## Gotchas ## Gotchas
- Readarr is in development (~v0.2.x). Some features may be incomplete. - Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
- Calibre integration is folder-based, not API-based. Readarr drops files into `/books` which Calibre-Web serves. the final tags). The pinned `amd64-1.0.980` tag exists precisely because
- Not all indexers have all formats (epub, mobi, pdf). Goodreads/StoryGraph metadata helps match editions. `:latest` does not exist for this image on LSIO.
- Calibre integration is folder-based, not API-based. Readarr drops files into
the books root which [[Calibre-Web]] serves.
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
metadata helps match editions while those providers still respond.
## See Also ## See Also
+48 -21
View File
@@ -1,40 +1,67 @@
---
type: reference
title: Redis (RomM Cache / Queue)
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
tags: [redis, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
# Redis (RomM Cache / Queue) # Redis (RomM Cache / Queue)
Redis 7 provides the caching and task queue backend for RomM on Dankserver. RomM 5.2.0 requires Redis (or Valkey) for its task queue. Redis 7 provides the caching and task-queue backend for [[RomM]] on Dankserver.
RomM requires Redis (or Valkey) for its task queue; this deployment runs
stock `redis:7-alpine` — no Valkey here, unlike the SearXNG cache on
Dankclaw ([[Valkey]]).
**Host:** Dankserver — **Docs:** https://redis.io/docs/ **Host:** Dankserver — container name `romm-redis`, unit `romm-redis.service`
(user-manager quadlet from `dankserver/romm/quadlet/romm-redis.container`).
## Facts ## Facts (verified from the live unit)
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `docker.io/redis:7-alpine` | | Image | `docker.io/library/redis:7-alpine` |
| Port | 6379 (internal, within `appnet`) | | Port | 6379 — internal only; no `PublishPort`, so it is unreachable from the host network by construction |
| Data volume | `/opt/romm/redis` | | Network | `dankserver_appnet` with `alias=redis` — that alias is what makes `REDIS_HOST=redis` in the RomM unit resolve |
| Auth | None (local-only, no password needed) | | Data volume | `/home/dank/Containers/romm/redis:/data` |
| Auth | None — safe precisely because the container is not published anywhere; only appnet members can reach it |
## Docker Compose ## Unit shape (as deployed)
```yaml ```ini
redis: [Container]
image: docker.io/redis:7-alpine ContainerName=romm-redis
container_name: romm-redis Image=docker.io/library/redis:7-alpine
restart: unless-stopped Network=dankserver_appnet:alias=redis
volumes: Volume=/home/dank/Containers/romm/redis:/data
- /opt/romm/redis:/data HealthCmd=redis-cli ping
healthcheck: HealthInterval=10s
test: ["CMD", "redis-cli", "ping"] HealthTimeout=5s
interval: 10s HealthRetries=3
timeout: 5s
retries: 3 [Service]
Restart=always
RestartSec=5
[Install]
WantedBy=default.target
``` ```
No `EnvironmentFile` — there is nothing secret in this unit; the auth-less
posture is enforced by network topology (alias-only reachability), not by a
password. If a future client ever needs Redis from outside the appnet, add
`PublishPort` **and** a `requirepass` in the same change — never just the port.
## Backup ## Backup
```bash ```bash
sudo rsync -a /opt/romm/redis /mnt/backup/romm-redis/ sudo rsync -a /home/dank/Containers/romm/redis /mnt/backup/romm-redis/
``` ```
Redis here is cache + queue state; losing it costs queued jobs and cache warm,
not the library (the database lives in [[MariaDB]]).
## See Also ## See Also
- [[RomM]] — service that uses this cache - [[RomM]] — service that uses this cache
- [[Valkey]] — the SearXNG-side equivalent on Dankclaw
+43 -26
View File
@@ -1,53 +1,70 @@
---
type: reference
title: Sonarr
description: Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
tags: [sonarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Sonarr # Sonarr
Sonarr manages TV shows in the *arr stack on Dankserver. It automatically searches, downloads, and organizes episodes from indexers into the Jellyfin media library. Sonarr manages TV shows in the *arr stack. The stack runs on **FrankBook-Pro**
(the Mac mini) as quadlet units inside its podman machine — not on Dankserver.
It searches, downloads, and organizes episodes from indexers into the Jellyfin
TV library.
**Host:** Dankserver — **Hostname:** `sonarr.dankserver.local` — **Docs:** https://wiki.servarr.com/sonarr **Host:** FrankBook-Pro (podman machine) — **Access:** direct port `8989` (no reverse-proxy vhost; the *arr stack is exempt by construction from the Anubis guard chain) — **Docs:** https://wiki.servarr.com/sonarr
## Facts ## Facts
| Item | Value | | Item | Value |
|------|-------| |------|-------|
| Image | `lscr.io/linuxserver/sonarr:latest` (v4 stable) | | Image | `lscr.io/linuxserver/sonarr:latest` (v4 line) |
| Port | 8989 (host) → 8989 (container) | | Port | 8989 (host) → 8989 (container) |
| Env | `PUID=0`, `PGID=0`, `TZ=America/Los_Angeles`, `UMASK_SET=022` |
| Config root | `/Volumes/COMPLETE/sonarr-data:/config` |
| Min RAM | 512 MB | | Min RAM | 512 MB |
| Metadata | TMDB / TVDB | | Metadata | TMDB / TVDB |
## Docker Compose ## Deployment (current)
```yaml Quadlet unit `sonarr.container` from this repo (`frankbook-pro/sonarr/quadlet/`)
sonarr: is installed inside the podman machine at
image: lscr.io/linuxserver/sonarr:latest `/home/core/.config/containers/systemd/sonarr.service` by the frankbook-pro
container_name: sonarr handler in `scripts/deploy-remote.sh`. Unit shape:
restart: unless-stopped
environment: ```ini
- PUID=1000 [Container]
- PGID=1000 ContainerName=sonarr
- TZ=America/New_York Image=lscr.io/linuxserver/sonarr:latest
- UMASK_SET=022 Environment=PUID=0
volumes: Environment=PGID=0
- /opt/sonarr/config:/config Environment=TZ=America/Los_Angeles
- /mnt/media/tv:/tv Environment=UMASK_SET=022
- /mnt/media/downloads:/downloads Volume=/Volumes/COMPLETE/sonarr-data:/config
ports: PublishPort=8989:8989
- "8989:8989"
networks: [appnet] [Service]
TimeoutStartSec=900
Restart=always
``` ```
## Configuration ## Configuration
1. Add Prowlarr as the indexer source. 1. Add Prowlarr as the indexer source (see [[Prowlarr]] — the feed route is `/{indexerId}/api`).
2. Add qBittorrent as the download client with the `tv` category. 2. Add qBittorrent as the download client with the `tv` category ([[qBittorrent]]).
3. Set quality profile, cutoff, and minimum/maximum episode sizes. 3. Set quality profile, cutoff, and minimum/maximum episode sizes.
4. Set root folder to `/tv`. 4. Set the root folder to the TV root under the COMPLETE volume (Sonarr imports
5. Add series and verify downloads. from where [[qBittorrent]] drops completed downloads).
5. Add a test series and verify the grab → download → import chain end-to-end.
## Gotchas ## Gotchas
- Sonarr v4 is the current stable. The v3 branch uses the `:v3` tag. - Sonarr v4 is the current stable. The v3 branch uses the `:v3` tag.
- Anime has different quality profiles and naming conventions than Standard series. - Anime has different quality profiles and naming conventions than Standard series.
- Missing episodes: set "Unmonitor Missing Episodes" to auto-unmonitor old episodes. - Missing episodes: set "Unmonitor Missing Episodes" to auto-unmonitor old episodes.
- Timezone-sensitive scheduler: ensure all containers use the same TZ. - Timezone-sensitive scheduler: every unit in the stack runs
`TZ=America/Los_Angeles`; keep them uniform.
## See Also ## See Also
@@ -4,9 +4,9 @@ A password-protected web site on Dankclaw lists the services that are up, with a
**Served by:** Caddy, `file_server` + `forward_auth` (Authentik SSO) — **Docs:** this page **Served by:** Caddy, `file_server` + `forward_auth` (Authentik SSO) — **Docs:** this page
## Status — 2026-08-23: deployed to production ## Status — deployed to production (2026-08-23), on the naked domain since 2026-08-24
The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the static assets to `/srv/homepage/`, installs the status checker (systemd timer, every 3 min, writes `status.json`), writes `deployed.js` (the last master deployment datetime), renders the Caddyfile (the shared basicauth hash from the host `.env`), reloads Caddy, and checks health. 146 tests pass. The Caddyfile auth uses a `route` block with `forward_auth` (see [Auth](#auth)); the caddy container mounts the host's `/srv/homepage` (not the repo source) so the generated files serve. The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the static assets to `/srv/homepage/`, installs the status checker (systemd timer, every 3 min, writes `status.json`), writes `deployed.js` (the last master deployment datetime), renders the Caddyfile (the shared basicauth hash from the host `.env`), reloads Caddy, and checks health. The Caddyfile auth uses a `route` block with `forward_auth` (see [Auth](#auth)); the caddy container mounts the host's `/srv/homepage` (not the repo source) so the generated files serve.
**Moved to the naked domain (2026-08-24):** the homepage now serves `franksdank.cloud` and `www.franksdank.cloud`. The old names `dankclaw.franksdank.cloud` and `dankserver.franksdank.cloud` redirect to it. Authentication moved from per-page basicauth to Authentik `forward_auth` (see [Authentik](Authentik.md)). **Moved to the naked domain (2026-08-24):** the homepage now serves `franksdank.cloud` and `www.franksdank.cloud`. The old names `dankclaw.franksdank.cloud` and `dankserver.franksdank.cloud` redirect to it. Authentication moved from per-page basicauth to Authentik `forward_auth` (see [Authentik](Authentik.md)).
@@ -28,7 +28,7 @@ The site has two pages. The root page lists the user services. The `/admin/` pag
```mermaid ```mermaid
graph TD graph TD
subgraph site["dankclaw.franksdank.cloud (Caddy)"] subgraph site["franksdank.cloud + www (Caddy)"]
ROOT["/ user services<br/>(shared password)"] ROOT["/ user services<br/>(shared password)"]
ADMIN["/admin/ admin services<br/>(operator password)"] ADMIN["/admin/ admin services<br/>(operator password)"]
HOW["/how/ how it is built<br/>(deferred)"] HOW["/how/ how it is built<br/>(deferred)"]
@@ -85,7 +85,9 @@ flowchart TD
**Admin services** (`/admin/`): Hermes, Grafana, Prometheus, Loki, Caddy, Pi-hole, Gluetun, the LLM fleet, and the low-level backends (mariadb, redis, Postgres, node-exporter, alloy). The backends show status only. They get a link only where a UI is meaningful. **Admin services** (`/admin/`): Hermes, Grafana, Prometheus, Loki, Caddy, Pi-hole, Gluetun, the LLM fleet, and the low-level backends (mariadb, redis, Postgres, node-exporter, alloy). The backends show status only. They get a link only where a UI is meaningful.
**Excluded for now:** the *arr stack (Radarr, Sonarr, Lidarr, Readarr, Prowlarr, qBittorrent). It is not started. It is a separate effort, possibly on a different host. **Excluded from the listing:** the *arr stack + qBittorrent (Prowlarr, Sonarr,
Radarr, Lidarr, Readarr, qB) — live on FrankBook-Pro but operator-only: no
vhosts, direct IP:port access only. See [[FrankBook-Pro]].
## Layout ## Layout
@@ -2,7 +2,9 @@
Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints. It feeds the Grafana dashboards. The `node-exporter` containers may add podman/cgroup metrics for full host visibility. Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints. It feeds the Grafana dashboards. The `node-exporter` containers may add podman/cgroup metrics for full host visibility.
**Host:** Dankclaw — **Hostname:** `prometheus.dankclaw.local` — **Port:** 9090 — **Docs:** https://prometheus.io/docs **Host:** Dankclaw — **Vhost:** `prometheus.franksdank.cloud` (guarded: Anubis →
Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
**Port:** 9090 — **Docs:** https://prometheus.io/docs
## Facts ## Facts
@@ -160,9 +160,9 @@ The DFlash draft is not a standalone model. It must pair with its target model.
## Deployment on Dankserver (NVIDIA RTX 3090) ## Deployment on Dankserver (NVIDIA RTX 3090)
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The Dankserver llama.cpp deployment (`frank-small`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-default-small` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only. > **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-small` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions: The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
@@ -195,7 +195,7 @@ docker run --rm --name vllm-qwen36-dflash \
## Deployment on Dankclaw (AMD Strix Halo gfx1151) ## Deployment on Dankclaw (AMD Strix Halo gfx1151)
`frank-default-large` on Dankclaw runs the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec is the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. See [[qwen3.8-spec-matrix]] and [[Qwen3.8 PP Deployment]] for the current state and the 2026-08-15 Vulkan0/ROCm0 high-context bench. `frank-large` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only. History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
@@ -14,7 +14,7 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
(b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx (b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not 302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
hold on the 8060S. See [[frank-default-large-benchmark]]. hold on the 8060S. See [[frank-large-benchmark]].
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
does not exist on this machine (checked 2026-08-15). GPU DPM is already does not exist on this machine (checked 2026-08-15). GPU DPM is already
`performance`, forced level `auto`. This item is N/A. `performance`, forced level `auto`. This item is N/A.
@@ -40,7 +40,7 @@ lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
different, driver-reported figure that has always been larger than the kernel different, driver-reported figure that has always been larger than the kernel
cap. cap.
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
limits documented in [[frank-default-large-benchmark]]. limits documented in [[frank-large-benchmark]].
## 1. Self-Hosted Agentic Tools ## 1. Self-Hosted Agentic Tools
@@ -98,7 +98,7 @@ All require root and a reboot for the modprobe change.
## Sources ## Sources
- [[frank-default-large-benchmark]] — measured backend/spec numbers - [[frank-large-benchmark]] — measured backend/spec numbers
- [[deployment]] — deploy and update process - [[deployment]] — deploy and update process
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS) - AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`, - llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
@@ -107,7 +107,7 @@ All require root and a reboot for the modprobe change.
## See Also ## See Also
- [[frank-default-large-benchmark]] - [[frank-large-benchmark]]
- [[DFlash DSpark Speculative Decoding]] - [[DFlash DSpark Speculative Decoding]]
- [[deployment]] - [[deployment]]
- [[index]] - [[index]]
@@ -38,7 +38,7 @@ Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host run
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-08-17**) | | Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-08-17**) |
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] | | Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
The NPU is retired; the 52625 port and the `frank-default-small` name now serve the CIRU ROCmFPX llama.cpp runtime (see [[ciru-activefpx-promptforge]]). The NPU is retired; the 52625 port and the `frank-small` name now serve the CIRU ROCmFPX llama.cpp runtime (see [[ciru-activefpx-promptforge]]).
### ROCm ### ROCm
@@ -93,7 +93,7 @@ cable between claw and server is required for 10G.
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM. - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget. - The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget.
- The active service is `frank-default-large` on port 7770 (moved from 52625 on 2026-08-22): CIRU ROCmFPX llama.cpp serving kingjones/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP (Q6 AGENT dense, 262144 ctx, standalone MTP Q4_0 head). See [[kingjones-strix-mtp]]. - The active service is `frank-large` on port 7770 (moved from 52625 on 2026-08-22): CIRU ROCmFPX llama.cpp serving kingjones/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP (Q6 AGENT dense, 262144 ctx, standalone MTP Q4_0 head). See [[kingjones-strix-mtp]].
- The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`. - The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18. - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 1.1 TB free disk space is ample for model storage. - 1.1 TB free disk space is ample for model storage.
@@ -60,7 +60,7 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
### Active Services ### Active Services
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q6_K) on port 7770, with expert tensors selected by `--override-tensor`; every full-attention block plus blocks 35–40 remain on CUDA. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]]. The active service on Dankserver is `frank-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q6_K) on port 7770, with expert tensors selected by `--override-tensor`; every full-attention block plus blocks 35–40 remain on CUDA. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
## Hardware Relevance ## Hardware Relevance
@@ -1,7 +1,7 @@
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology. > **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
> See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and > See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and
> [deployment.md](deployment.md) for the active configuration. > [deployment.md](deployment.md) for the active configuration.
> The `frank-default-small` name and port 52625 now belong to the vLLM > The `frank-small` name and port 52625 now belong to the vLLM
> service on Dankclaw (nameistoken Quark W8A8); this page is archive. > service on Dankclaw (nameistoken Quark W8A8); this page is archive.
--- ---
@@ -19,7 +19,7 @@ timestamp: 2026-08-16
# FastFlowLM NPU Inference # FastFlowLM NPU Inference
FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-default-small` (port 52625) serves in production. See Promotion. FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-small` (port 52625) serves in production. See Promotion.
## Verdict (2026-08-16) ## Verdict (2026-08-16)
@@ -168,7 +168,7 @@ Proposed role: a small, always-on, low-power endpoint for triage, summarization,
- `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`. - `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
- `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below). - `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below).
- `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot. - `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
- The `llama-server` (frank-default-large, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16. - The `llama-server` (frank-large, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU) ## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
@@ -186,7 +186,7 @@ Long-context prefill is not yet measured (runs 1-3 used under 200 prompt tokens)
## Model Bakeoff (measured 2026-08-16, 4 models) ## Model Bakeoff (measured 2026-08-16, 4 models)
All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-default-large) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom). All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-large) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) | | Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------| |-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
@@ -204,21 +204,21 @@ Notes:
## Promotion (2026-08-16) ## Promotion (2026-08-16)
`frank-default-small` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-default-small.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-default-small.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch. `frank-small` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-small.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-small.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
### Boot policy (2026-08-17) ### Boot policy (2026-08-17)
The `frank-default-small` unit is disabled for boot. Only the PP pair (`frank-default-large` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw: The `frank-small` unit is disabled for boot. Only the PP pair (`frank-large` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw:
``` ```
sudo ln -sf /home/dank/vllmScripts/scripts/frank-default-small.service /etc/systemd/system/frank-default-small.service sudo ln -sf /home/dank/vllmScripts/scripts/frank-small.service /etc/systemd/system/frank-small.service
sudo systemctl daemon-reload sudo systemctl daemon-reload
sudo systemctl enable frank-default-small sudo systemctl enable frank-small
``` ```
Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom). Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
The PP split was renamed `frank-pp-head` → `frank-default-large` (2026-08-16, commit `48142f5`). The PP split was renamed `frank-pp-head` → `frank-large` (2026-08-16, commit `48142f5`).
Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it. Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it.
@@ -277,7 +277,7 @@ graph TD
Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window. Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`. 1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`.
2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-default-small` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-default-small' is not supported.` Each such request also triggered the unknown-tag fallback below. 2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-small` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-small' is not supported.` Each such request also triggered the unknown-tag fallback below.
3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load. 3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load.
Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16. Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16.
@@ -314,7 +314,7 @@ pi sessions failed against this endpoint with `Stream ended without finish_reaso
- **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step. - **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step.
- **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this. - **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this.
- **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags. - **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags.
- **NPU production role (2026-08-16).** `frank-default-small` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded. - **NPU production role (2026-08-16).** `frank-small` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded.
## Citations ## Citations
@@ -26,7 +26,7 @@ conclusions.
| Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` | | Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` |
| FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` | | FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` |
| KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed | | KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed |
| ubatch | 256 | `UBATCH_SIZE` in `frank-default-large.env` | | ubatch | 256 | `UBATCH_SIZE` in `frank-large.env` |
| MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` | | MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` |
| Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s | | Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s |
| Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends | | Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends |
@@ -189,7 +189,7 @@ For the 2-node pipeline-parallel split (RTX 3090 24 GB + Strix Halo 96 GB):
- **Kept:** Qwen official config.json (HF) — exact architecture params (64 layers, 5120 hidden, 4 KV heads, head_dim 256) [8](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/config.json) - **Kept:** Qwen official config.json (HF) — exact architecture params (64 layers, 5120 hidden, 4 KV heads, head_dim 256) [8](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/config.json)
- **Kept:** bartowski/Qwen_Qwen3.6-27B-GGUF — Q8_0 at 29.12 GB, imatrix calibration [2](https://huggingface.co/bartowski/Qwen_Qwen3.6-27B-GGUF) - **Kept:** bartowski/Qwen_Qwen3.6-27B-GGUF — Q8_0 at 29.12 GB, imatrix calibration [2](https://huggingface.co/bartowski/Qwen_Qwen3.6-27B-GGUF)
- **Kept:** Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp — DFlash 2B draft GGUF model [4](https://huggingface.co/Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp) - **Kept:** Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp — DFlash 2B draft GGUF model [4](https://huggingface.co/Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp)
- **Kept:** frank-default-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](../frank-default-large-benchmark.md) - **Kept:** frank-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](../frank-large-benchmark.md)
- **Kept:** a4lg/Qwen3.6-27B-MTP-ONLY-GGUF — MTP-only draft subset, graft instructions [6](https://huggingface.co/a4lg/Qwen3.6-27B-MTP-ONLY-GGUF) - **Kept:** a4lg/Qwen3.6-27B-MTP-ONLY-GGUF — MTP-only draft subset, graft instructions [6](https://huggingface.co/a4lg/Qwen3.6-27B-MTP-ONLY-GGUF)
- **Kept:** llama.cpp tools/rpc/README.md — RPC backend, weight/KV split, remote host setup [7](https://github.com/ggerganov/llama.cpp/blob/master/tools/rpc/README.md) - **Kept:** llama.cpp tools/rpc/README.md — RPC backend, weight/KV split, remote host setup [7](https://github.com/ggerganov/llama.cpp/blob/master/tools/rpc/README.md)
- **Kept:** Unsloth Qwen3.6-27B-MTP-GGUF — MTP command-line usage for llama.cpp [9](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF) - **Kept:** Unsloth Qwen3.6-27B-MTP-GGUF — MTP command-line usage for llama.cpp [9](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF)
@@ -11,7 +11,7 @@ morning-of runbook. Research background:
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68) ## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
This split is the production `frank-default-large` service. Current This split is the production `frank-large` service. Current
state, verified against the live journal and output smoke tests state, verified against the live journal and output smoke tests
(2026-08-17, load 02:06): (2026-08-17, load 02:06):
@@ -32,17 +32,17 @@ state, verified against the live journal and output smoke tests
Vulkan). Vulkan).
- Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but - Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
stable; all allocations are upfront). stable; all allocations are upfront).
- Unit `frank-default-large.service`, launched through - Unit `frank-large.service`, launched through
`serve-pp-head.sh` + `frank-default-large.env`. `serve-pp-head.sh` + `frank-large.env`.
## Service enablement (2026-08-17) ## Service enablement (2026-08-17)
The PP pair is the only inference service enabled for boot. Both units The PP pair is the only inference service enabled for boot. Both units
(`frank-default-large` on Dankclaw, `frank-pp-worker` on Dankserver) are (`frank-large` on Dankclaw, `frank-pp-worker` on Dankserver) are
enabled. The NPU unit (`frank-default-small` on Dankclaw) and all retired enabled. The NPU unit (`frank-small` on Dankclaw) and all retired
units are disabled. Restart order: worker first, then head. The Dankserver units are disabled. Restart order: worker first, then head. The Dankserver
helper `frank-restart-small.sh` is obsolete (it restarts the removed helper `frank-restart-small.sh` is obsolete (it restarts the removed
`frank-default-small` unit); restart the worker with `frank-small` unit); restart the worker with
`sudo systemctl restart frank-pp-worker`. The Dankclaw helper `sudo systemctl restart frank-pp-worker`. The Dankclaw helper
`frank-restart-large.sh` (sync + cache drop + restart) is correct for the `frank-restart-large.sh` (sync + cache drop + restart) is correct for the
head. `deploy.sh dankserver` is broken until retargeted. See head. `deploy.sh dankserver` is broken until retargeted. See
@@ -164,9 +164,9 @@ llama-server b10433 ggml-rpc-server b10433
|------|---------|---------| |------|---------|---------|
| `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) | | `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
| `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build | | `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
| `scripts/frank-default-large.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename | | `scripts/frank-large.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
| `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") | | `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
| `scripts/frank-default-large.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` | | `scripts/frank-large.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
| `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) | | `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
| `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` | | `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
@@ -220,7 +220,7 @@ FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh stop-pp
FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
``` ```
Config variants: edit `scripts/frank-default-large.env` on the claw (git commit + Config variants: edit `scripts/frank-large.env` on the claw (git commit +
pull if the bench runs from the repo), then re-run `start` + `bench`: pull if the bench runs from the repo), then re-run `start` + `bench`:
- No spec baseline: `SPEC_NONE=1` - No spec baseline: `SPEC_NONE=1`
@@ -76,6 +76,6 @@ graph LR
## See Also ## See Also
- [[DFlash DSpark Speculative Decoding]] - [[DFlash DSpark Speculative Decoding]]
- [[frank-default-small-benchmark]] - [[frank-small-benchmark]]
- [[Dankserver System Specs]] - [[Dankserver System Specs]]
- [[Dankclaw System Specs]] - [[Dankclaw System Specs]]
@@ -47,7 +47,7 @@ The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB
## Prerequisites ## Prerequisites
> **Production note:** Dankclaw's primary service now runs vLLM-ROCm with the Ling-3.0-flash CIRU INT4 model (see `scripts/frank-default-large.service`). This page documents the alternative vLLM FP8 deployment of Qwen3.6-27B. > **Production note:** Dankclaw's primary service now runs vLLM-ROCm with the Ling-3.0-flash CIRU INT4 model (see `scripts/frank-large.service`). This page documents the alternative vLLM FP8 deployment of Qwen3.6-27B.
Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding: Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:
@@ -114,7 +114,7 @@ export HF_HOME=/mnt/entropy/huggingface
### Active Service Conflict ### Active Service Conflict
The retired `frank-default-small` service (llama.cpp) used about 22 GB of the 24 GB VRAM. The current PP worker holds ~23.8 GiB. The `vllm-awq-qwen36` (Podman) and `vllm-qwen36` units are disabled. Before starting any research service, verify that the GPU is free: The retired `frank-small` service (llama.cpp) used about 22 GB of the 24 GB VRAM. The current PP worker holds ~23.8 GiB. The `vllm-awq-qwen36` (Podman) and `vllm-qwen36` units are disabled. Before starting any research service, verify that the GPU is free:
``` ```
nvidia-smi # verify GPU is free nvidia-smi # verify GPU is free
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw title: agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
description: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF served by the charlie12345/ROCmFPX llama.cpp runtime (build-qwen38). 87.06 GiB Q4_0_ROCMFP4_FAST imatrix (v2 joined PLE layout), own MTP draft head (ROCmFP4-FAST, 2.28 GiB). frank-default-large. description: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF served by the charlie12345/ROCmFPX llama.cpp runtime (build-qwen38). 87.06 GiB Q4_0_ROCMFP4_FAST imatrix (v2 joined PLE layout), own MTP draft head (ROCmFP4-FAST, 2.28 GiB). frank-large.
tags: tags:
- llama.cpp - llama.cpp
- rocm - rocm
@@ -26,7 +26,7 @@ timestamp: 2026-08-29
> `--flash-attn on` work on the charlie build-qwen38 tree — all still > `--flash-attn on` work on the charlie build-qwen38 tree — all still
> true for that binary + model pair, which remains the rollback target. > true for that binary + model pair, which remains the rollback target.
`frank-default-large` (port 7770) served `frank-large` (port 7770) served
`agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF` (main, `agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF` (main,
`Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf`, the `v2/` joined layout) `Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf`, the `v2/` joined layout)
with the author's own MTP draft head with the author's own MTP draft head
@@ -166,7 +166,7 @@ best: 32.4 t/s, 0.612 acceptance; adaptive p-split measured worse).
## Config files ## Config files
- Env: `scripts/frank-default-large.env` - Env: `scripts/frank-large.env`
- Serve: `scripts/serve-frank-large.sh` - Serve: `scripts/serve-frank-large.sh`
- Chat template: `scripts/chat-template.jinja` (unchanged, Qwen-Sharp) - Chat template: `scripts/chat-template.jinja` (unchanged, Qwen-Sharp)
- Fork patch: `scripts/patches/qwen4exp-qsa-f32-mask.patch` (re-apply after - Fork patch: `scripts/patches/qwen4exp-qsa-f32-mask.patch` (re-apply after
@@ -175,10 +175,10 @@ best: 32.4 t/s, 0.612 acceptance; adaptive p-split measured worse).
## Swap procedure ## Swap procedure
```bash ```bash
sudo systemctl stop frank-default-large sudo systemctl stop frank-large
# edit scripts/frank-default-large.env + scripts/serve-frank-large.sh, commit, push # edit scripts/frank-large.env + scripts/serve-frank-large.sh, commit, push
# on Dankclaw: git pull --ff-only && scripts/deploy.sh dankclaw --service frank-default-large # on Dankclaw: git pull --ff-only && scripts/deploy.sh dankclaw --service frank-large
sudo systemctl start frank-default-large # or via deploy.sh sudo systemctl start frank-large # or via deploy.sh
curl http://192.168.0.143:7770/health curl http://192.168.0.143:7770/health
``` ```
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: CIRU ActiveFPX PromptForge on Dankclaw title: CIRU ActiveFPX PromptForge on Dankclaw
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-large service. description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-large service.
tags: tags:
- llama.cpp - llama.cpp
- rocm - rocm
@@ -14,12 +14,12 @@ timestamp: 2026-08-22
# CIRU ActiveFPX PromptForge on Dankclaw # CIRU ActiveFPX PromptForge on Dankclaw
> **Active 2026-08-22.** `frank-default-large` serves this v3 model again > **Active 2026-08-22.** `frank-large` serves this v3 model again
> after a same-day Kairic Edge IU4 experiment was reverted (operator > after a same-day Kairic Edge IU4 experiment was reverted (operator
> decision). Commit `c60156f` restored the v3 script pair. See > decision). Commit `c60156f` restored the v3 script pair. See
> [[kairic-edge]] for the retired experiment reference. > [[kairic-edge]] for the retired experiment reference.
`frank-default-large` (port 7770, moved from 52625 on 2026-08-22) serves `frank-large` (port 7770, moved from 52625 on 2026-08-22) serves
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21. previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
@@ -135,7 +135,7 @@ graph TD
C --> D[Build llama-server] C --> D[Build llama-server]
D --> E[Verify --list-devices shows gfx1151] D --> E[Verify --list-devices shows gfx1151]
E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj] E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj]
F --> G[Deploy frank-default-large] F --> G[Deploy frank-large]
G --> H[Health check + chat completion] G --> H[Health check + chat completion]
``` ```
@@ -150,8 +150,8 @@ against the model card before the service restart.
## Service ## Service
- Unit: `frank-default-large.service`, port 7770 - Unit: `frank-large.service`, port 7770
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env` - Script: `scripts/serve-frank-large.sh`, env `scripts/frank-large.env`
- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before - The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
the system ROCm. `ROCM_PATH` points at the devel SDK. the system ROCm. `ROCM_PATH` points at the devel SDK.
@@ -5,7 +5,7 @@
--- ---
type: reference type: reference
title: Dankserver VRAM and RAM Growth Under Load title: Dankserver VRAM and RAM Growth Under Load
description: Investigation of the recurring cuBLAS crash on frank-default-small. What grows, what does not, and what to do next. description: Investigation of the recurring cuBLAS crash on frank-small. What grows, what does not, and what to do next.
tags: tags:
- dankserver - dankserver
- llamacpp - llamacpp
@@ -18,7 +18,7 @@ tags:
# Dankserver VRAM and RAM Growth Under Load # Dankserver VRAM and RAM Growth Under Load
This page records the investigation of the recurring cuBLAS crash on This page records the investigation of the recurring cuBLAS crash on
`frank-default-small` (llama.cpp, RTX 3090 24 GiB). The service aborts `frank-small` (llama.cpp, RTX 3090 24 GiB). The service aborts
with `CUDA error: an unsupported value or parameter was passed to the with `CUDA error: an unsupported value or parameter was passed to the
function` in `cublasSgemm_v2` at `ggml-cuda.cu:1541` during function` in `cublasSgemm_v2` at `ggml-cuda.cu:1541` during
`update_slots` -> `decode` -> `process_ubatch`. It crashed on 2026-08-13 `update_slots` -> `decode` -> `process_ubatch`. It crashed on 2026-08-13
@@ -116,7 +116,7 @@ Verified 2026-08-14 after deploy: flash attention enabled, warmup clean, decode
## Mitigation Options ## Mitigation Options
The documentation for the benchmark ([[frank-default-small-benchmark]]) The documentation for the benchmark ([[frank-small-benchmark]])
prescribes the first option for this crash class. None of these change prescribes the first option for this crash class. None of these change
the model. the model.
@@ -135,6 +135,6 @@ PR #25649 (slot checkpoints freed on prompt clear, merged), issue #26119
## See Also ## See Also
- [[frank-default-small-benchmark]] - [[frank-small-benchmark]]
- [[Dankserver System Specs]] - [[Dankserver System Specs]]
- [[deployment]] - [[deployment]]
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-default-small. description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-small.
tags: tags:
- llamacpp - llamacpp
- cuda - cuda
@@ -14,12 +14,12 @@ timestamp: 2026-08-24
# dealign Ornith-1.5-35B-A3B CRACK on Dankserver # dealign Ornith-1.5-35B-A3B CRACK on Dankserver
> **Active 2026-08-25.** `frank-default-small` serves this model. It > **Active 2026-08-25.** `frank-small` serves this model. It
> replaced the CRACK Q8_0 file on the same upstream llama.cpp binary (b10121). > replaced the CRACK Q8_0 file on the same upstream llama.cpp binary (b10121).
> The Q6_K file was downloaded from the model repository and validated with > The Q6_K file was downloaded from the model repository and validated with
> `llama-gguf` before installation. > `llama-gguf` before installation.
`frank-default-small` (port 7770) serves `frank-small` (port 7770) serves
`Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf` from `dealignai` on Hugging Face. It is `Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf` from `dealignai` on Hugging Face. It is
a **MoE** model (256 experts, 8 active) with a hybrid GatedDeltaNet (SSM) + a **MoE** model (256 experts, 8 active) with a hybrid GatedDeltaNet (SSM) +
attention architecture, abliterated (uncensored). The GGUF also contains a attention architecture, abliterated (uncensored). The GGUF also contains a
+24 -18
View File
@@ -19,8 +19,9 @@ SearXNG, Caddy, or another container service does **not** restart inference.
| Target | Host | Service | Port | | Target | Host | Service | Port |
|---|---|---|---:| |---|---|---|---:|
| Dankserver | `dank@dankserver` | `frank-default-small` | 7770 | | Dankserver | `dank@dankserver` | `frank-small` | 7770 |
| Dankclaw | `dank@dankclaw` | `frank-default-large` | 7770 | | Dankclaw | `dank@dankclaw` | `frank-large` | 7770 |
| Dankclaw | `dank@dankclaw` | `frank-npu` (FLM on the XDNA2 NPU) | 7771 |
## Scoped workflow ## Scoped workflow
@@ -38,34 +39,39 @@ graph LR
Run the unified deployer from the repository root: Run the unified deployer from the repository root:
```bash ```bash
./scripts/deploy.sh dankserver --service frank-default-small ./scripts/deploy.sh dankserver --service frank-small
./scripts/deploy.sh dankclaw --service frank-default-large ./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --dry-run ./scripts/deploy.sh dankclaw --dry-run
``` ```
Without `--service`, `scripts/deploy-plan.py` maps changed files to the service Without `--service`, `scripts/deploy-plan.py` maps changed files to the service
on the selected host. The following files select the corresponding service: on the selected host. The following files select the corresponding service:
- `scripts/frank-default-small.env`, `frank-default-small.service`, and - `scripts/frank-small.env`, `frank-small.service`, and
`serve-frank-small.sh` select `frank-default-small` on Dankserver. `serve-frank-small.sh` select `frank-small` on Dankserver.
- `scripts/frank-default-large.env`, `frank-default-large.service`, and - `scripts/frank-large.env`, `frank-large.service`, and
`serve-frank-large.sh` select `frank-default-large` on Dankclaw. `serve-frank-large.sh` select `frank-large` on Dankclaw.
- `scripts/frank-npu.env`, `frank-npu.service`,
and `serve-frank-npu.sh` select `frank-npu`
on Dankclaw.
- `scripts/chat-template.jinja` selects the LLM on the selected host only. - `scripts/chat-template.jinja` selects the LLM on the selected host only.
- Documentation, tests, deployment tooling, and unrelated container changes - Documentation, tests, deployment tooling, and unrelated container changes
do not select either inference service. do not select either inference service.
The target-side `scripts/deploy-remote.sh` relinks the selected unit into The target-side `scripts/deploy-remote.sh` relinks the selected unit into
`/etc/systemd/system/`, reloads systemd, enables it, restarts it, and waits for `/etc/systemd/system/`, reloads systemd, enables it, restarts it, and waits for
`http://127.0.0.1:7770/health`. The wait budget is 20 minutes because cold model that service's health endpoint (`http://127.0.0.1:7770/health` for the two
loads can be slow. A failed handler does not advance its per-service state. inference services; `/v1/version` for the NPU sight+sound endpoint — FLM has no
`/health` route). The wait budget is 20 minutes because cold model loads can be
slow. A failed handler does not advance its per-service state.
## Unit and artifact paths ## Unit and artifact paths
Both units point into the stable target checkout: Both units point into the stable target checkout:
```text ```text
/home/dank/serverAppConfigs/scripts/frank-default-small.service /home/dank/serverAppConfigs/scripts/frank-small.service
/home/dank/serverAppConfigs/scripts/frank-default-large.service /home/dank/serverAppConfigs/scripts/frank-large.service
``` ```
The unit files and serve scripts validate their model/runtime artifacts before The unit files and serve scripts validate their model/runtime artifacts before
@@ -74,8 +80,8 @@ the unit absolute paths and the deployment tests in the same change.
## NPU unit ## NPU unit
`frank-default-npu` is an inactive, historical Dankclaw endpoint. It is `frank-npu` is an inactive, historical Dankclaw endpoint. It is
relinked only when explicitly selected with `--service frank-default-npu`; the relinked only when explicitly selected with `--service frank-npu`; the
large-model handler never restarts it. This is intentional: the NPU is not part large-model handler never restarts it. This is intentional: the NPU is not part
of the active inference path and should not be disturbed by a large-model of the active inference path and should not be disturbed by a large-model
change. change.
@@ -99,16 +105,16 @@ change in that service's inputs.
If the LLM handler fails after the target pull: If the LLM handler fails after the target pull:
1. Inspect `journalctl -u frank-default-small -n 200` or 1. Inspect `journalctl -u frank-small -n 200` or
`journalctl -u frank-default-large -n 200`. `journalctl -u frank-large -n 200`.
2. Confirm the model files, custom runtime binary, and chat template are 2. Confirm the model files, custom runtime binary, and chat template are
readable by `dank`. readable by `dank`.
3. Retry only the affected service: 3. Retry only the affected service:
```bash ```bash
./scripts/deploy.sh dankserver --service frank-default-small ./scripts/deploy.sh dankserver --service frank-small
# or # or
./scripts/deploy.sh dankclaw --service frank-default-large ./scripts/deploy.sh dankclaw --service frank-large
``` ```
4. Verify the matching health endpoint before declaring recovery. 4. Verify the matching health endpoint before declaring recovery.
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: DFlash2 on Dankclaw (frank-default-large) title: DFlash2 on Dankclaw (frank-large)
description: ARCHIVED - unsloth Qwen3.8-27B-UD-Q8_K_XL served by upstream llama.cpp PR #27342 with a z-lab DFlash2 drafter on the Vulkan backend. Retired 2026-08-21 for speed; kept as a runbook. description: ARCHIVED - unsloth Qwen3.8-27B-UD-Q8_K_XL served by upstream llama.cpp PR #27342 with a z-lab DFlash2 drafter on the Vulkan backend. Retired 2026-08-21 for speed; kept as a runbook.
tags: tags:
- llama.cpp - llama.cpp
@@ -13,14 +13,14 @@ tags:
timestamp: 2026-08-21 timestamp: 2026-08-21
--- ---
# DFlash2 on Dankclaw (frank-default-large) — ARCHIVED # DFlash2 on Dankclaw (frank-large) — ARCHIVED
> **Archived 2026-08-21.** This DFlash2 Vulkan deployment was retired the > **Archived 2026-08-21.** This DFlash2 Vulkan deployment was retired the
> same day it shipped because it was slower than the CIRU runtime it > same day it shipped because it was slower than the CIRU runtime it
> replaced. `frank-default-large` is back on CIRU. This page stays as the > replaced. `frank-large` is back on CIRU. This page stays as the
> runbook and provenance for the DFlash2 attempt. > runbook and provenance for the DFlash2 attempt.
`frank-default-large` (port 52625) briefly served `frank-large` (port 52625) briefly served
`unsloth/Qwen3.8-27B-UD-Q8_K_XL` with a DFlash2 drafter from z-lab. The `unsloth/Qwen3.8-27B-UD-Q8_K_XL` with a DFlash2 drafter from z-lab. The
runtime was upstream llama.cpp PR #27342 built with `GGML_VULKAN=ON`. runtime was upstream llama.cpp PR #27342 built with `GGML_VULKAN=ON`.
@@ -114,14 +114,14 @@ graph TD
C --> D[Build llama-server] C --> D[Build llama-server]
D --> E[Verify Vulkan0 device] D --> E[Verify Vulkan0 device]
E --> F[Add DFlash2 drafter] E --> F[Add DFlash2 drafter]
F --> G[Swap frank-default-large] F --> G[Swap frank-large]
G --> H[Health check + completion] G --> H[Health check + completion]
``` ```
## Service ## Service
- Unit: `frank-default-large.service`, port 52625 - Unit: `frank-large.service`, port 52625
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env` - Script: `scripts/serve-frank-large.sh`, env `scripts/frank-large.env`
- Binary: `/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server` - Binary: `/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server`
### Launch profile ### Launch profile
+10 -10
View File
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw title: caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
description: caf-eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF (Q4_0_ROCMI4, embedded NextN/MTP head) served by the charlie12345/ROCmFPX llama.cpp runtime (ROCmI4 W4A4) on the Strix Halo iGPU. frank-default-large. description: caf-eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF (Q4_0_ROCMI4, embedded NextN/MTP head) served by the charlie12345/ROCmFPX llama.cpp runtime (ROCmI4 W4A4) on the Strix Halo iGPU. frank-large.
tags: tags:
- llama.cpp - llama.cpp
- rocm - rocm
@@ -15,12 +15,12 @@ timestamp: 2026-08-25
# caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw # caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
> **Active 2026-08-25.** `frank-default-large` serves this model. It replaced > **Active 2026-08-25.** `frank-large` serves this model. It replaced
> the kingjones777 ROCmFP4 STRIX MTP payload on the same port and the same > the kingjones777 ROCmFP4 STRIX MTP payload on the same port and the same
> Strix Halo iGPU, running the new charlie12345/ROCmFPX ROCmI4 W4A4 build. See > Strix Halo iGPU, running the new charlie12345/ROCmFPX ROCmI4 W4A4 build. See
> [[kingjones-strix-mtp]] for the retired page. > [[kingjones-strix-mtp]] for the retired page.
`frank-default-large` (port 7770) serves `frank-large` (port 7770) serves
`cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (published under the public `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (published under the public
`eemin` namespace) from the **charlie12345/ROCmFPX** llama.cpp fork built with `eemin` namespace) from the **charlie12345/ROCmFPX** llama.cpp fork built with
the opt-in `ROCmI4 W4A4` path. The model is `Q4_0_ROCMI4`: signed four-bit the opt-in `ROCmI4 W4A4` path. The model is `Q4_0_ROCMI4`: signed four-bit
@@ -127,8 +127,8 @@ path's greedy decisions; it does not remove the W4A4 activation approximation.
### Service ### Service
- Unit: `frank-default-large.service`, port 7770 - Unit: `frank-large.service`, port 7770
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env` - Script: `scripts/serve-frank-large.sh`, env `scripts/frank-large.env`
## Tuning MTP ## Tuning MTP
@@ -142,14 +142,14 @@ Decode throughput depends on the ubatch (per-eval token batch). Run the sweep
on 8k contexts, 512 → 4096 in steps of 512, with a fixed batch: on 8k contexts, 512 → 4096 in steps of 512, with a fixed batch:
``` ```
sudo systemctl stop frank-default-large sudo systemctl stop frank-large
./scripts/bench-ubatch.sh ./scripts/bench-ubatch.sh
sudo systemctl start frank-default-large sudo systemctl start frank-large
``` ```
The script stops `frank-default-large` for a clean GPU, boots one server The script stops `frank-large` for a clean GPU, boots one server
instance per ubatch on a scratch port, reads authoritative `print_timing` tok/s, instance per ubatch on a scratch port, reads authoritative `print_timing` tok/s,
and restarts the service. See [[frank-default-large-benchmark]]. and restarts the service. See [[frank-large-benchmark]].
## Known Issues ## Known Issues
@@ -183,4 +183,4 @@ The ubatch sweep above gives the production ubatch choice.
- [[kingjones-strix-mtp]] — the retired ROCmFP4 predecessor on this slot - [[kingjones-strix-mtp]] — the retired ROCmFP4 predecessor on this slot
- [[ciru-activefpx-promptforge]] — the retired v3 CIRU PromptForge page - [[ciru-activefpx-promptforge]] — the retired v3 CIRU PromptForge page
- [[Dankclaw System Specs]] — the box - [[Dankclaw System Specs]] — the box
- [[frank-default-large-benchmark]] — backend/spec benchmark history - [[frank-large-benchmark]] — backend/spec benchmark history
@@ -2,10 +2,10 @@
> See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and > See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and
> [deployment.md](deployment.md) for the active configuration. > [deployment.md](deployment.md) for the active configuration.
# frank-default-large — Backend and Spec Benchmark # frank-large — Backend and Spec Benchmark
This page records the benchmarks that chose the runtime for This page records the benchmarks that chose the runtime for
`frank-default-large` on Dankclaw. Every number came from the live `frank-large` on Dankclaw. Every number came from the live
llama.cpp build on that machine (upstream, b10133+). llama.cpp build on that machine (upstream, b10133+).
## Summary ## Summary
+11 -11
View File
@@ -1,6 +1,6 @@
--- ---
type: configuration type: configuration
title: frank-default-sightsound Runbook title: frank-npu Runbook
description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving Qwen3.8 Distilled Heretic vision+text pack on port 7771, including the IOMMU/SVA root cause that took it down at first deploy. description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving Qwen3.8 Distilled Heretic vision+text pack on port 7771, including the IOMMU/SVA root cause that took it down at first deploy.
tags: tags:
- flm - flm
@@ -8,17 +8,17 @@ tags:
- xdna2 - xdna2
- npu - npu
- dankclaw - dankclaw
- frank-default-sightsound - frank-npu
timestamp: 2026-09-01 timestamp: 2026-09-01
--- ---
# frank-default-sightsound — FLM NPU sight+sound Runbook # frank-npu — FLM NPU sight+sound Runbook
`frank-default-sightsound` is the third inference endpoint on Dankclaw: a `frank-npu` is the third inference endpoint on Dankclaw: a
FastFlowLM (FLM) v1.0.1 server (`/usr/bin/flm`, dpkg package `fastflowlm`) FastFlowLM (FLM) v1.0.1 server (`/usr/bin/flm`, dpkg package `fastflowlm`)
running on the **XDNA2 NPU** (50 TOPS). It serves *sight* (vision) — running on the **XDNA2 NPU** (50 TOPS). It serves *sight* (vision) —
which the iGPU llama.cpp fleet ([[kingjones-strix-rocmfp4]], which the iGPU llama.cpp fleet ([[kingjones-strix-rocmfp4]],
`frank-default-large`) does not do. The model-selection tradeoff: the only `frank-large`) does not do. The model-selection tradeoff: the only
NPU2 pack family on this box with BOTH a vision and an audio tower is the NPU2 pack family on this box with BOTH a vision and an audio tower is the
Gemma4 E4B IT pack; the uncensored candidate (`qwythos:9b`, 8.94 GB, 262k ctx) Gemma4 E4B IT pack; the uncensored candidate (`qwythos:9b`, 8.94 GB, 262k ctx)
is text-only and fails the hard modality requirement, so this endpoint trades is text-only and fails the hard modality requirement, so this endpoint trades
@@ -29,9 +29,9 @@ is text-only and fails the hard modality requirement, so this endpoint trades
| Field | Value | | Field | Value |
|-------|-------| |-------|-------|
| Runtime | FLM v1.0.1 (`/usr/bin/flm`, dpkg `fastflowlm`) on the XDNA2 NPU | | Runtime | FLM v1.0.1 (`/usr/bin/flm`, dpkg `fastflowlm`) on the XDNA2 NPU |
| Service | `frank-default-sightsound.service` (systemd system unit, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`) | | Service | `frank-npu.service` (systemd system unit, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`) |
| Files | `scripts/frank-default-sightsound.env`, `scripts/serve-frank-default-sightsound.sh`, `scripts/frank-default-sightsound.service` | | Files | `scripts/frank-npu.env`, `scripts/serve-frank-npu.sh`, `scripts/frank-npu.service` |
| Deploy | `scripts/deploy.sh dankclaw --service frank-default-sightsound` (relink + daemon-reload + enable + restart; health gate polls `/v1/version`, 300 s cap — FLM has **no `/health` route**) | | Deploy | `scripts/deploy.sh dankclaw --service frank-npu` (relink + daemon-reload + enable + restart; health gate polls `/v1/version`, 300 s cap — FLM has **no `/health` route**) |
| Port | `7771` on `0.0.0.0` (52625/52626 left free for ad-hoc `flm serve` experiments) | | Port | `7771` on `0.0.0.0` (52625/52626 left free for ad-hoc `flm serve` experiments) |
| Power mode | `FLM_PMODE=turbo` — no other tenant uses the NPU on this host (the large model runs on the iGPU), so the top power mode costs no neighbour anything; drop to `performance` if iGPU power-budget contention ever shows | | Power mode | `FLM_PMODE=turbo` — no other tenant uses the NPU on this host (the large model runs on the iGPU), so the top power mode costs no neighbour anything; drop to `performance` if iGPU power-budget contention ever shows |
| Context | `FLM_CTX_LEN=131072 (128K; verified max on this box; native context length.) | | Context | `FLM_CTX_LEN=131072 (128K; verified max on this box; native context length.) |
@@ -57,8 +57,8 @@ clients choose, send requests with the desired `model` field.
**To swap the active model** (changes what the service loads at start): **To swap the active model** (changes what the service loads at start):
``` ```
sed -i 's/^FLM_MODEL=.*/FLM_MODEL=<tag>/' scripts/frank-default-sightsound.env sed -i 's/^FLM_MODEL=.*/FLM_MODEL=<tag>/' scripts/frank-npu.env
./scripts/deploy.sh dankclaw --service frank-default-sightsound ./scripts/deploy.sh dankclaw --service frank-npu
``` ```
**To switch per-request** (service stays loaded, client sends the tag): **To switch per-request** (service stays loaded, client sends the tag):
@@ -119,7 +119,7 @@ chain, all live-verified on the host:
3. Fix: replace `iommu=off` with **`iommu=pt`** in `GRUB_CMDLINE_LINUX_DEFAULT` 3. Fix: replace `iommu=off` with **`iommu=pt`** in `GRUB_CMDLINE_LINUX_DEFAULT`
on `/etc/default/grub`, `update-grub`, reboot. Passthrough mode keeps the on `/etc/default/grub`, `update-grub`, reboot. Passthrough mode keeps the
iGPU default domain identity-mapped (zero DMA-translation overhead for iGPU default domain identity-mapped (zero DMA-translation overhead for
`frank-default-large`) while enabling IOMMU hardware + PASID, which is what `frank-large`) while enabling IOMMU hardware + PASID, which is what
the NPU driver's SVA bind requires — PASID domains are paged regardless of the NPU driver's SVA bind requires — PASID domains are paged regardless of
the default-domain type. the default-domain type.
4. Post-boot verification: `/sys/kernel/iommu_groups/` populated (30 groups), 4. Post-boot verification: `/sys/kernel/iommu_groups/` populated (30 groups),
@@ -8,15 +8,15 @@ The old scripts moved to `old/scripts/` in this repository.
| Service | Host | Runtime | Model | Port | | Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------| |---------|------|---------|-------|------|
| frank-default-large | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 | | frank-large | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
| frank-default-small | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 | | frank-small | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
The RTX 3090 runs the whole 27B model standalone. No RPC worker, no The RTX 3090 runs the whole 27B model standalone. No RPC worker, no
tensor split, no NPU. tensor split, no NPU.
## Models ## Models
### frank-default-large — HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF:IQ4_XS ### frank-large — HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF:IQ4_XS
- Base: Qwen3.8-27B (dense 27B, 64 layers: 48 GatedDeltaNet + 16 - Base: Qwen3.8-27B (dense 27B, 64 layers: 48 GatedDeltaNet + 16
gated-attention, hidden 5120, 24 Q heads / 4 KV heads, head dim 256, gated-attention, hidden 5120, 24 Q heads / 4 KV heads, head dim 256,
@@ -31,7 +31,7 @@ tensor split, no NPU.
`--spec-draft-ngl all`, `--spec-draft-p-min 0`) then ngram-mod fallback `--spec-draft-ngl all`, `--spec-draft-p-min 0`) then ngram-mod fallback
(48/64/24). (48/64/24).
### frank-default-small — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8 ### frank-small — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
- Base: Qwen3.6-35B-A3B (hybrid MoE, 256 experts top-8, ~3B active, 40 - Base: Qwen3.6-35B-A3B (hybrid MoE, 256 experts top-8, ~3B active, 40
layers: 10 blocks of 3 DeltaNet + 1 Attention, native 262144 ctx). layers: 10 blocks of 3 DeltaNet + 1 Attention, native 262144 ctx).
@@ -46,7 +46,7 @@ tensor split, no NPU.
instead of the Qwen3 XML structure). The Quark quant follows the format instead of the Qwen3 XML structure). The Quark quant follows the format
cleanly; swapped in the evening of 2026-08-17. cleanly; swapped in the evening of 2026-08-17.
## Context Math (frank-default-large, RTX 3090) ## Context Math (frank-large, RTX 3090)
KV per token: 16 attention layers x 4 KV heads x 256 dim. KV per token: 16 attention layers x 4 KV heads x 256 dim.
K q8_0 = 1088 B/token, V f16 = 2048 B/token, total 50176 B/token. K q8_0 = 1088 B/token, V f16 = 2048 B/token, total 50176 B/token.
@@ -63,11 +63,11 @@ Budget (MiB): 24576 total - 14982 weights - 861 draft - 1024 fixed
Production choice: **131072 (128K)** — the maximum that keeps about Production choice: **131072 (128K)** — the maximum that keeps about
1.4 GiB headroom for checkpoints. Lower `CTX_SIZE` in 1.4 GiB headroom for checkpoints. Lower `CTX_SIZE` in
`scripts/frank-default-large.env` to 122880 for ~1.8 GiB headroom. `scripts/frank-large.env` to 122880 for ~1.8 GiB headroom.
## Install Runbook ## Install Runbook
### Dankserver (frank-default-large) ### Dankserver (frank-large)
1. Stop the old worker: `sudo systemctl stop frank-pp-worker`. 1. Stop the old worker: `sudo systemctl stop frank-pp-worker`.
2. Download the model and draft to `/home/dank/models/hauhaucs-qwen38/`. 2. Download the model and draft to `/home/dank/models/hauhaucs-qwen38/`.
@@ -87,17 +87,17 @@ Production choice: **131072 (128K)** — the maximum that keeps about
measured 95 t/s vs 1300+ t/s at 8K on the 3090). measured 95 t/s vs 1300+ t/s at 8K on the 3090).
4. Symlink the unit, reload, enable, restart: 4. Symlink the unit, reload, enable, restart:
``` ```
sudo ln -sf /home/dank/vLLM/scripts/frank-default-large.service /etc/systemd/system/ sudo ln -sf /home/dank/vLLM/scripts/frank-large.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-default-large sudo systemctl daemon-reload && sudo systemctl enable frank-large
sudo systemctl restart frank-default-large sudo systemctl restart frank-large
``` ```
5. Verify: `curl -sf http://127.0.0.1:7770/health` and a chat completion. 5. Verify: `curl -sf http://127.0.0.1:7770/health` and a chat completion.
6. Confirm VRAM headroom with `nvidia-smi`. 6. Confirm VRAM headroom with `nvidia-smi`.
### Dankclaw (frank-default-small) ### Dankclaw (frank-small)
1. Stop the old services: `sudo systemctl stop frank-default-large 1. Stop the old services: `sudo systemctl stop frank-large
frank-default-small` (retired PP head and NPU endpoint). frank-small` (retired PP head and NPU endpoint).
2. Install uv and create the venv: 2. Install uv and create the venv:
``` ```
curl -LsSf https://astral.sh/uv/install.sh | sh curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -122,9 +122,9 @@ Production choice: **131072 (128K)** — the maximum that keeps about
vision encoder falls back to torch SDPA and engine init OOMs. vision encoder falls back to torch SDPA and engine init OOMs.
5. Symlink the unit, reload, enable, restart: 5. Symlink the unit, reload, enable, restart:
``` ```
sudo ln -sf /home/dank/vllmScripts/scripts/frank-default-small.service /etc/systemd/system/ sudo ln -sf /home/dank/vllmScripts/scripts/frank-small.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-default-small sudo systemctl daemon-reload && sudo systemctl enable frank-small
sudo systemctl restart frank-default-small sudo systemctl restart frank-small
``` ```
6. Verify: `curl -sf http://127.0.0.1:52625/health` and a chat completion. 6. Verify: `curl -sf http://127.0.0.1:52625/health` and a chat completion.
@@ -150,7 +150,7 @@ on gfx1151; newer vLLM releases may already include it.
Three changes landed after the initial deployment: Three changes landed after the initial deployment:
### 1. Tool calling on frank-default-small (HTTP 400 fix) ### 1. Tool calling on frank-small (HTTP 400 fix)
Clients (pi) send `tool_choice: "auto"`. vLLM rejects it with HTTP 400 Clients (pi) send `tool_choice: "auto"`. vLLM rejects it with HTTP 400
unless the server enables tool calling. Fixes applied: unless the server enables tool calling. Fixes applied:
@@ -175,7 +175,7 @@ model behavior, which motivated the swap.
### 2. Model swap: Avesed INT8 W8A8 -> Quark W8A8 INT8 ### 2. Model swap: Avesed INT8 W8A8 -> Quark W8A8 INT8
`frank-default-small` now serves `frank-small` now serves
`nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8` (AMD Quark 0.11.1, ~35 GB, `nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8` (AMD Quark 0.11.1, ~35 GB,
vLLM-native expert layout). The Avesed compressed-tensors quant's vLLM-native expert layout). The Avesed compressed-tensors quant's
Qwen3-XML tool fidelity was poor. Verified with the Quark model: plain Qwen3-XML tool fidelity was poor. Verified with the Quark model: plain
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: frank-default-small Benchmark — ncmoe and Batch Tuning title: frank-small Benchmark — ncmoe and Batch Tuning
description: llama-bench sweep of MoE CPU offload (ncmoe) and batch/ubatch on the RTX 3090 for Qwen3.6-35B-A3B Q6_K_P. description: llama-bench sweep of MoE CPU offload (ncmoe) and batch/ubatch on the RTX 3090 for Qwen3.6-35B-A3B Q6_K_P.
tags: tags:
- benchmark - benchmark
@@ -10,11 +10,11 @@ tags:
- dankserver - dankserver
--- ---
# frank-default-small Benchmark — ncmoe and Batch/Ubatch Tuning # frank-small Benchmark — ncmoe and Batch/Ubatch Tuning
This page records a llama-bench sweep on Dankserver (RTX 3090, 24 GiB) for This page records a llama-bench sweep on Dankserver (RTX 3090, 24 GiB) for
the HauhauCS Qwen3.6-35B-A3B Q6_K_P model. The sweep decided the final the HauhauCS Qwen3.6-35B-A3B Q6_K_P model. The sweep decided the final
`--n-cpu-moe` and `--ubatch-size` values for `frank-default-small`. `--n-cpu-moe` and `--ubatch-size` values for `frank-small`.
Run date: 2026-08-13. Build: llama.cpp `555881ebc` (10121). CUDA, 7 threads, Run date: 2026-08-13. Build: llama.cpp `555881ebc` (10121). CUDA, 7 threads,
`--cache-type-k q8_0 --cache-type-v f16`, `--fit-target 0` (KV auto-sized to `--cache-type-k q8_0 --cache-type-v f16`, `--fit-target 0` (KV auto-sized to
@@ -105,7 +105,7 @@ service runs 700416 cells (13,680 MiB).
## Chosen State ## Chosen State
> **Live drift (2026-08-22 audit):** the deployed `frank-default-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation. > **Live drift (2026-08-22 audit):** the deployed `frank-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
Prefill speed is the priority, so the state keeps the top prefill tier and Prefill speed is the priority, so the state keeps the top prefill tier and
moves one expert layer to CPU for stability: moves one expert layer to CPU for stability:
@@ -6,7 +6,7 @@ tags:
- freetoken - freetoken
- benchmark - benchmark
- dankserver - dankserver
- frank-default-small - frank-small
- qwen3.6 - qwen3.6
timestamp: 2026-09-02 timestamp: 2026-09-02
--- ---
@@ -90,5 +90,5 @@ Tiel-Coder finetune.
## See Also ## See Also
- [[freetoken-frank-default-small]] — production runbook - [[freetoken-frank-small]] — production runbook
- [[dealign-ornith-crack]] — the prior llama.cpp deployment (retired) - [[dealign-ornith-crack]] — the prior llama.cpp deployment (retired)
@@ -1,20 +1,20 @@
--- ---
type: configuration type: configuration
title: FreeToken frank-default-small Runbook title: FreeToken frank-small Runbook
description: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-default-small on Dankserver (RTX 3090). description: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-small on Dankserver (RTX 3090).
tags: tags:
- freetoken - freetoken
- dankserver - dankserver
- frank-default-small - frank-small
- qwen3.6 - qwen3.6
- moe - moe
- fp8 - fp8
timestamp: 2026-08-30 timestamp: 2026-08-30
--- ---
# FreeToken frank-default-small Runbook # FreeToken frank-small Runbook
The `frank-default-small` service on Dankserver is served by The `frank-small` service on Dankserver is served by
**FreeToken v0.1.2** — an edge-native MoE serving engine that keeps **FreeToken v0.1.2** — an edge-native MoE serving engine that keeps
expert weights in host RAM with an LRU GPU cache. Model history on this expert weights in host RAM with an LRU GPU cache. Model history on this
runtime: official `Qwen/Qwen3.6-35B-A3B-FP8` (since 2026-09-02) → runtime: official `Qwen/Qwen3.6-35B-A3B-FP8` (since 2026-09-02) →
@@ -130,8 +130,8 @@ modelopt block-fp8 remains the only FP8 path that loads.
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `scripts/serve-frank-small.sh` | Serve script (launches `ft serve`) | | `scripts/serve-frank-small.sh` | Serve script (launches `ft serve`) |
| `scripts/frank-default-small.env` | Tunables (model path, pool sizes) | | `scripts/frank-small.env` | Tunables (model path, pool sizes) |
| `scripts/frank-default-small.service` | Systemd unit | | `scripts/frank-small.service` | Systemd unit |
## Limitations ## Limitations
@@ -150,7 +150,7 @@ modelopt block-fp8 remains the only FP8 path that loads.
- **`model` field is required** in `/v1/chat/completions` bodies — - **`model` field is required** in `/v1/chat/completions` bodies —
FreeToken rejects requests without it with a 422 (`Field required`), FreeToken rejects requests without it with a 422 (`Field required`),
unlike llama.cpp's server which defaults to the loaded model. Send unlike llama.cpp's server which defaults to the loaded model. Send
`"model": "frank-default-small"`. `"model": "frank-small"`.
- Load on this swap took well under the old llama.cpp numbers: - Load on this swap took well under the old llama.cpp numbers:
weights → experts (parallel build) → CUDA graph capture → Uvicorn weights → experts (parallel build) → CUDA graph capture → Uvicorn
in ~2 minutes; the 2026-08-30 swap verified ~40 tok/s decode on a in ~2 minutes; the 2026-08-30 swap verified ~40 tok/s decode on a
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: ik_llama.cpp on Dankserver (Ornith Q8_0) title: ik_llama.cpp on Dankserver (Ornith Q8_0)
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags. description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-small on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
tags: tags:
- ikllama - ikllama
- llamacpp - llamacpp
@@ -16,7 +16,7 @@ timestamp: 2026-08-26
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK) # ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
> **Active 2026-08-26.** `frank-default-small` switched from upstream > **Active 2026-08-26.** `frank-small` switched from upstream
> llama.cpp (b10121) to `ikawrakow/ik_llama.cpp` (commit `b166e269`) on the > llama.cpp (b10121) to `ikawrakow/ik_llama.cpp` (commit `b166e269`) on the
> RTX 3090. This page documents the build, the mandatory FA build flags, and > RTX 3090. This page documents the build, the mandatory FA build flags, and
> the MoE / CPU-offload runtime arguments. > the MoE / CPU-offload runtime arguments.
@@ -101,8 +101,8 @@ a hard compile error.
## Runtime configuration ## Runtime configuration
Files: `scripts/serve-frank-small.sh`, `scripts/frank-default-small.env`, Files: `scripts/serve-frank-small.sh`, `scripts/frank-small.env`,
`scripts/frank-default-small.service` in the repo. The service just calls `scripts/frank-small.service` in the repo. The service just calls
the serve script; the env file holds all tunables. the serve script; the env file holds all tunables.
### KV cache: K default f16 / V q6_0 + Hadamard (V only) ### KV cache: K default f16 / V q6_0 + Hadamard (V only)
@@ -241,7 +241,7 @@ apply.
The previous runtime `upstream llama.cpp b10121` and its binary The previous runtime `upstream llama.cpp b10121` and its binary
(`/home/dank/llama.cpp/build/bin/llama-server`) are **left untouched**. To (`/home/dank/llama.cpp/build/bin/llama-server`) are **left untouched**. To
rollback to the old stack, change `LLAMA_SERVER=` in rollback to the old stack, change `LLAMA_SERVER=` in
`scripts/frank-default-small.env` back to that path and `scripts/frank-small.env` back to that path and
`CACHE_TYPE_K/V` back to `f16` with Hadamard off, and re-enable the ngram `CACHE_TYPE_K/V` back to `f16` with Hadamard off, and re-enable the ngram
spec / `--spec-type` flags. The model file is not changed either way. spec / `--spec-type` flags. The model file is not changed either way.
+20 -17
View File
@@ -1,9 +1,9 @@
# LLM Fleet — Knowledge Base # LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on two local machines Documentation for the LLM inference fleet on two local machines
(2026-08-30): FreeToken v0.1.2 on Dankserver (frank-default-small, (2026-08-30): FreeToken v0.1.2 on Dankserver (frank-small,
Huihui-Qwen3.6-35B-A3B-abliterated-FP8) and kingjones30/ROCmFPX Huihui-Qwen3.6-35B-A3B-abliterated-FP8) and kingjones30/ROCmFPX
llama.cpp on Dankclaw (frank-default-large, kingjones777 llama.cpp on Dankclaw (frank-large, kingjones777
Qwen3.8-Flash-Next ROCmFP4-STRIX). Both services run the Qwen-Sharp chat template. Qwen3.8-Flash-Next ROCmFP4-STRIX). Both services run the Qwen-Sharp chat template.
> 2026-08-22: the former `vllmScripts` repository is merged into the > 2026-08-22: the former `vllmScripts` repository is merged into the
@@ -18,7 +18,7 @@ graph LR
subgraph Dankserver[NVIDIA Dankserver - Ubuntu 26.04] subgraph Dankserver[NVIDIA Dankserver - Ubuntu 26.04]
A[RTX 3090 24GB + 60GB RAM] A[RTX 3090 24GB + 60GB RAM]
B2[FreeToken v0.1.2 B2[FreeToken v0.1.2
frank-default-small, port 7770] frank-small, port 7770]
C[Huihui-Qwen3.6-35B-A3B-abliterated-FP8 C[Huihui-Qwen3.6-35B-A3B-abliterated-FP8
block-fp8, 20% MoE residency, 453k KV / 4 streams] block-fp8, 20% MoE residency, 453k KV / 4 streams]
A --> B2 --> C A --> B2 --> C
@@ -27,7 +27,7 @@ block-fp8, 20% MoE residency, 453k KV / 4 streams]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[gfx1151 iGPU 64 GiB unified memory] E2[gfx1151 iGPU 64 GiB unified memory]
E[kingjones30/ROCmFPX build E[kingjones30/ROCmFPX build
frank-default-large, port 7770] frank-large, port 7770]
F[kingjones777 Qwen3.8-Flash-Next F[kingjones777 Qwen3.8-Flash-Next
ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE] ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
E2 --> E --> F E2 --> E --> F
@@ -46,18 +46,18 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
## Active Deployment ## Active Deployment
- [[freetoken-frank-default-small]] — the active frank-default-small - [[freetoken-frank-small]] — the active frank-small
runbook: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B- runbook: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-
abliterated-FP8 on Dankserver (swapped 2026-08-30; same format/pools) abliterated-FP8 on Dankserver (swapped 2026-08-30; same format/pools)
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep - [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090 (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers - [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[kingjones-strix-rocmfp4]] — ACTIVE frank-default-large runbook: - [[kingjones-strix-rocmfp4]] — ACTIVE frank-large runbook:
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, on the card's pinned (56B MoE + 51.2B PLE), single 113.47 GiB GGUF, on the card's pinned
kingjones30/ROCmFPX fork (swapped 2026-08-30) kingjones30/ROCmFPX fork (swapped 2026-08-30)
- [[agentionai-fast-imatrix]] — retired frank-default-large runbook: - [[agentionai-fast-imatrix]] — retired frank-large runbook:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB
Q4_0_ROCMFP4_FAST imatrix, v2 joined PLE) on build-qwen38 Q4_0_ROCMFP4_FAST imatrix, v2 joined PLE) on build-qwen38
(2026-08-29 → 2026-08-30); rollback binary + payload retained (2026-08-29 → 2026-08-30); rollback binary + payload retained
@@ -66,9 +66,9 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model - [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
on Dankclaw (retired) on Dankclaw (retired)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, retired 2026-09-02) Dankserver (frank-small, retired 2026-09-02)
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for - [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for
frank-default-small (2026-08-26): q8_0 + Hadamard KV cache, frank-small (2026-08-26): q8_0 + Hadamard KV cache,
split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags
- [[kingjones-strix-mtp]] — the former Q6 AGENT runbook (restored 2026-09-01 - [[kingjones-strix-mtp]] — the former Q6 AGENT runbook (restored 2026-09-01
after Q8_0 failure, then superseded by the STRIX deployment on 2026-08-27) after Q8_0 failure, then superseded by the STRIX deployment on 2026-08-27)
@@ -76,9 +76,10 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
on Dankclaw (superseded 2026-08-24 by the kingjones ROCmFP4 model) on Dankclaw (superseded 2026-08-24 by the kingjones ROCmFP4 model)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment - [[kairic-edge]] — the retired Kairic Edge IU4 experiment
(deployed and reverted 2026-08-22) (deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the RTX 3090 box (frank-default-small) - [[Dankserver System Specs]] — the RTX 3090 box (frank-small)
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box (frank-default-large) - [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large,
- [[frank-default-sightsound]] — ACTIVE FLM NPU sight+sound runbook on served on the pinned kingjones30/ROCmFPX llama.cpp fork since 2026-08-30)
- [[frank-npu]] — ACTIVE FLM NPU sight+sound runbook on
Dankclaw: Gemma4 E4B vision+audio pack on the XDNA2 NPU (FLM v1.0.1, port Dankclaw: Gemma4 E4B vision+audio pack on the XDNA2 NPU (FLM v1.0.1, port
7771, whole-registry serving, per-request model routing), incl. the 7771, whole-registry serving, per-request model routing), incl. the
IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash
@@ -101,8 +102,10 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism - [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table - [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware - [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware
notes and sizing (the historical family; frank-default-small now serves notes and sizing (this architecture family is what frank-small
Ornith-1.5-35B-A3B, see [[dealign-ornith-crack]]) serves today in the Huihui abliterated FP8 variant — see
[[freetoken-frank-small]]; the Ornith-1.5-35B-A3B CRACK checkpoint
that preceded it is archived at [[dealign-ornith-crack]])
## Memory Management ## Memory Management
@@ -115,7 +118,7 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
- [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker - [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP) - [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP)
- [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint - [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head - [[frank-large-benchmark]] — backend/spec benchmark behind the old head
- [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash - [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash
- [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting - [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting
- [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU - [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU
@@ -134,7 +137,7 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
## Related ## Related
- [Project knowledge base](../index.md) — the Podman app stacks on the same - [Project knowledge base](../index.md) — the Podman app stacks on the same
two hosts. OpenWebUI consumes the `:7770` endpoint on Dankserver. The two hosts. OpenWebUI (Dankclaw) consumes the on-host `frank-large`
Gitea instance on Dankclaw hosts this repository. endpoint at `:7770`. The Gitea instance on Dankclaw hosts this repository.
- [Deployment Flow](../Architecture/Deployment%20Flow.md) — the service-scoped - [Deployment Flow](../Architecture/Deployment%20Flow.md) — the service-scoped
git-based deploy with inference isolation git-based deploy with inference isolation
+4 -4
View File
@@ -17,10 +17,10 @@ timestamp: 2026-08-22
> **Retired 2026-08-22.** This experiment was deployed, benchmarks were > **Retired 2026-08-22.** This experiment was deployed, benchmarks were
> collected, and the service reverted to v3 the same day (operator > collected, and the service reverted to v3 the same day (operator
> decision). `frank-default-large` serves v3 again. This page is kept as > decision). `frank-large` serves v3 again. This page is kept as
> the retired experiment's build runbook and reference. > the retired experiment's build runbook and reference.
`frank-default-large` (port 52625) briefly served `frank-large` (port 52625) briefly served
`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX
llama.cpp runtime. The runtime routes qualified prompt and multi-token llama.cpp runtime. The runtime routes qualified prompt and multi-token
verification shapes through the RDNA 3.5 native verification shapes through the RDNA 3.5 native
@@ -215,7 +215,7 @@ service restart confirms the fresh tree, then it is a deletion candidate
The service runs the pinned runner profile plus the local chat The service runs the pinned runner profile plus the local chat
deviations below. Tunables live in deviations below. Tunables live in
`scripts/frank-default-large.env`; flags in `scripts/serve-frank-large.sh`. `scripts/frank-large.env`; flags in `scripts/serve-frank-large.sh`.
See [[deployment]] for the update process. See [[deployment]] for the update process.
| Setting | Value | Origin | | Setting | Value | Origin |
@@ -301,7 +301,7 @@ ubatches do not. Production runs the runner's qualified 2048/512.
The v3 payload and binary stay on the machine for rollback. The v3 payload and binary stay on the machine for rollback.
1. Stop the service: `sudo systemctl stop frank-default-large` 1. Stop the service: `sudo systemctl stop frank-large`
2. Restore the v3 scripts: 2. Restore the v3 scripts:
`git revert 4c43839 7aee1df 40d50fe` (or checkout the script pair from `git revert 4c43839 7aee1df 40d50fe` (or checkout the script pair from
commit `132e147`) commit `132e147`)
@@ -1,11 +1,11 @@
# frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT # frank-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
**Status:** Deployed on Dankclaw (since 2026-08-25). Serving is **BLOCKED by **Status:** Deployed on Dankclaw (since 2026-08-25). Serving is **BLOCKED by
the `set_rows` build bug** — the model loads but crashes in the `set_rows` build bug** — the model loads but crashes in
`ggml_compute_forward_set_rows` before it can serve. All config goals are met; `ggml_compute_forward_set_rows` before it can serve. All config goals are met;
a build fix is required for the model to run. a build fix is required for the model to run.
**Alias:** `frank-default-large`, port **7770**. **Alias:** `frank-large`, port **7770**.
The large service serves the kingjones `Qwen3.8-27B-ROCmFPX-Q8_0-AGENT` GGUF The large service serves the kingjones `Qwen3.8-27B-ROCmFPX-Q8_0-AGENT` GGUF
from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` with a from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` with a
@@ -26,7 +26,7 @@ fit in the lifted **114 GiB TTM/GTT** budget (see the 114 GiB change in
| Reasoning | on, `deepseek`, budget `-1`, `--reasoning-format none` | | Reasoning | on, `deepseek`, budget `-1`, `--reasoning-format none` |
| Chat template | Sharp (repo-tracked), `enable_thinking` + `preserve_thinking` on | | Chat template | Sharp (repo-tracked), `enable_thinking` + `preserve_thinking` on |
| Host path | `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` | | Host path | `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` |
| Tunables | `scripts/frank-default-large.env` | | Tunables | `scripts/frank-large.env` |
| Serve script | `scripts/serve-frank-large.sh` | | Serve script | `scripts/serve-frank-large.sh` |
## Serving status: the `set_rows` build bug (blocks serving) ## Serving status: the `set_rows` build bug (blocks serving)
@@ -107,7 +107,7 @@ dl mtp-Qwen3.8-27B-Q4_0.gguf
``` ```
The serve script checks each artifact is present and readable before launch The serve script checks each artifact is present and readable before launch
(`scripts/frank-default-large.env` -> the four `MODEL_*` files plus the chat (`scripts/frank-large.env` -> the four `MODEL_*` files plus the chat
template). A missing file fails the artifact check with `missing or template). A missing file fails the artifact check with `missing or
unreadable required artifact`. unreadable required artifact`.
@@ -116,7 +116,7 @@ unreadable required artifact`.
`scripts/serve-frank-large.sh` builds the `llama-server` call from the env: `scripts/serve-frank-large.sh` builds the `llama-server` call from the env:
``` ```
/dev/llama-server -m <main> --alias frank-default-large /dev/llama-server -m <main> --alias frank-large
--host 0.0.0.0 --port 7770 --jinja --host 0.0.0.0 --port 7770 --jinja
-dev ROCm0 -ngl 999 -np 1 -c 262144 -dev ROCm0 -ngl 999 -np 1 -c 262144
-b 512 -ub 512 -fa on -b 512 -ub 512 -fa on
@@ -197,7 +197,7 @@ and a 28 GB + 262k-KV load stalls. The GTT page-limit change is per the
## Env tunables ## Env tunables
`scripts/frank-default-large.env`. Serving: `CTX_SIZE=262144`, `scripts/frank-large.env`. Serving: `CTX_SIZE=262144`,
`PARALLEL=1`, `BATCH_SIZE=512`, `UBATCH_SIZE=512`, `NUM_THREADS=16`, `PARALLEL=1`, `BATCH_SIZE=512`, `UBATCH_SIZE=512`, `NUM_THREADS=16`,
`THREAD_BATCH=32`, `CACHE_TYPE_K=f16`, `CACHE_TYPE_V=turbo4`, `THREAD_BATCH=32`, `CACHE_TYPE_K=f16`, `CACHE_TYPE_V=turbo4`,
`CTX_CHECKPOINTS=32`, `CACHE_RAM=8192`. Reasoning: `REASONING=on`, `CTX_CHECKPOINTS=32`, `CACHE_RAM=8192`. Reasoning: `REASONING=on`,
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-default-large. description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-large.
tags: tags:
- llama.cpp - llama.cpp
- rocm - rocm
@@ -16,13 +16,13 @@ timestamp: 2026-08-24
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw # kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
> **Active again 2026-09-01.** The kingjones ROCmFP4 STRIX MTP payload is > **Active again 2026-09-01.** The kingjones ROCmFP4 STRIX MTP payload is
> restored as `frank-default-large` after the Q8_0 AGENT experiment. The Q8_0 > restored as `frank-large` after the Q8_0 AGENT experiment. The Q8_0
> model was retired because it crashed before serving. The Q6 AGENT model, > model was retired because it crashed before serving. The Q6 AGENT model,
> standalone MTP draft, Q8 projector, and PromptForge runtime are now active. > standalone MTP draft, Q8 projector, and PromptForge runtime are now active.
> This page documents the active model and its runbook. > This page documents the active model and its runbook.
`frank-default-large` (port 7770) serves `frank-large` (port 7770) serves
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging `Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
@@ -1,7 +1,7 @@
--- ---
type: reference type: reference
title: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw title: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw
description: ACTIVE frank-default-large runbook. kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the card's pinned kingjones30/ROCmFPX fork. 177B params (56B MoE + 51.2B PLE streamed via mmap), single 113.47 GiB GGUF, 401408 ctx x 2 slots. description: ACTIVE frank-large runbook. kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the card's pinned kingjones30/ROCmFPX fork. 177B params (56B MoE + 51.2B PLE streamed via mmap), single 113.47 GiB GGUF, 401408 ctx x 2 slots.
tags: tags:
- llama.cpp - llama.cpp
- rocm - rocm
@@ -23,10 +23,10 @@ timestamp: 2026-08-30
> decoding is OFF (the card ships no spec flags; the Flash-Next MTP heads > decoding is OFF (the card ships no spec flags; the Flash-Next MTP heads
> do not load on this loader family — see MTP Draft Head). > do not load on this loader family — see MTP Draft Head).
`frank-default-large` (port 7770) serves `frank-large` (port 7770) serves
`Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf` (single file, 113.47 GiB) `Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf` (single file, 113.47 GiB)
from `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/` from `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/`
via `scripts/serve-frank-large.sh` + `scripts/frank-default-large.env` via `scripts/serve-frank-large.sh` + `scripts/frank-large.env`
and the Qwen-Sharp chat template. and the Qwen-Sharp chat template.
## Model Architecture ## Model Architecture
@@ -178,7 +178,7 @@ cmake --build build --target llama-server llama-quantize -j16
## Serving Profile ## Serving Profile
Deployed args (`scripts/frank-default-large.env` — our validated Deployed args (`scripts/frank-large.env` — our validated
profile, unchanged from the FAST deployment except the binary and model): profile, unchanged from the FAST deployment except the binary and model):
``` ```
@@ -272,9 +272,9 @@ The file is 113.47 GiB. Allow ~20-30 minutes for download depending on network.
1. Build the runtime (see Runtime build) BEFORE touching the service. 1. Build the runtime (see Runtime build) BEFORE touching the service.
2. Point the model file's directory at the canonical path (a plain `mv` 2. Point the model file's directory at the canonical path (a plain `mv`
— never delete a 113 GiB payload to re-fetch it). — never delete a 113 GiB payload to re-fetch it).
3. Edit `scripts/frank-default-large.env` (+ serve script defaults) via 3. Edit `scripts/frank-large.env` (+ serve script defaults) via
git: commit, push, `./scripts/deploy.sh dankclaw --service git: commit, push, `./scripts/deploy.sh dankclaw --service
frank-default-large`. frank-large`.
4. Verify health: `curl http://192.168.0.143:7770/health` and a real 4. Verify health: `curl http://192.168.0.143:7770/health` and a real
`/v1/chat/completions` probe. `/v1/chat/completions` probe.
@@ -284,7 +284,7 @@ binary and the agentionai payload remain on disk.
### Config files ### Config files
- Env: `scripts/frank-default-large.env` - Env: `scripts/frank-large.env`
- Serve: `scripts/serve-frank-large.sh` - Serve: `scripts/serve-frank-large.sh`
- Chat template: `scripts/chat-template.jinja` - Chat template: `scripts/chat-template.jinja`
@@ -1,10 +1,6 @@
# Ling-3.0-flash Q6_K Trial on Dankclaw # Ling-3.0-flash Q6_K Trial on Dankclaw
> **RETIRED 2026-08-17.** The `frank-large-ling` unit is disabled under the boot policy. `frank-default-large` now runs the Qwen3.8-27B PP head, not the Qwen3.6-27B config described in the (obsolete) switch-back section below. > **RETIRED 2026-08-17.** The `frank-large-ling` unit is disabled under the boot policy. At this page's retirement date `frank-large` ran the Qwen3.8-27B PP head (since superseded by the STRIX deployment — see [[kingjones-strix-rocmfp4]]), not the Qwen3.6-27B config described in the (obsolete) switch-back section below.
This page records the Ling-3.0-flash trial that temporarily replaced
`frank-default-large` on Dankclaw. It replaces the Qwen3.6-27B service on
port 7770 under the same API alias.
## Status ## Status
@@ -73,22 +69,22 @@ point varies (385-1588 tokens). Single streams never freeze.
`SPEC_TYPE=none`, `PARALLEL=2`. All concurrent streams complete (EOS). `SPEC_TYPE=none`, `PARALLEL=2`. All concurrent streams complete (EOS).
Speed cost: ~10-13 t/s under contention vs ~28-32 t/s single-stream with Speed cost: ~10-13 t/s under contention vs ~28-32 t/s single-stream with
MTP. Fallback chain if spec-off regresses: A) `PARALLEL=1` keep MTP, MTP. Fallback chain if spec-off regresses: A) `PARALLEL=1` keep MTP,
B) switch back to Qwen3.6-27B (frank-default-large, MTP n_max 3). B) switch back to Qwen3.6-27B (frank-large, MTP n_max 3).
## Files ## Files
- `scripts/frank-large-ling.service` - systemd unit. - `scripts/frank-large-ling.service` - systemd unit.
- `scripts/serve-ling-flash.sh` - serve script (fork binary, Vulkan, - `scripts/serve-ling-flash.sh` - serve script (fork binary, Vulkan,
draft-mtp). draft-mtp).
- `scripts/frank-default-large.ling.env` - tunables. - `scripts/frank-large.ling.env` - tunables.
Passwordless management is installed: Passwordless management is installed:
`/usr/local/sbin/frank-restart-ling.sh` and `/etc/sudoers.d/frank-ling`. `/usr/local/sbin/frank-restart-ling.sh` and `/etc/sudoers.d/frank-ling`.
## Switch Back (obsolete) ## Switch Back (obsolete)
The switch-back commands below are obsolete. `frank-default-large` now The switch-back commands below are obsolete. At retirement time
runs the Qwen3.8-27B pipeline-parallel head. To restore a Ling trial, `frank-large` ran the Qwen3.8-27B pipeline-parallel head. To restore a Ling trial,
re-enable `frank-large-ling` from the repo unit re-enable `frank-large-ling` from the repo unit
(`scripts/frank-large-ling.service`) and serve script (`scripts/frank-large-ling.service`) and serve script
(`scripts/serve-ling-flash.sh`), then start it on a non-conflicting port. (`scripts/serve-ling-flash.sh`), then start it on a non-conflicting port.
@@ -107,6 +103,6 @@ re-enable `frank-large-ling` from the repo unit
## See Also ## See Also
- [[frank-default-large-benchmark]] - [[frank-large-benchmark]]
- [[DFlash DSpark Speculative Decoding]] - [[DFlash DSpark Speculative Decoding]]
- [[deployment]] - [[deployment]]
@@ -6,5 +6,5 @@ Dankclaw. The deployment never shipped. The actual Ling work used the
llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]]. llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]].
The trial service (`frank-large-ling`) is retired and disabled under the The trial service (`frank-large-ling`) is retired and disabled under the
boot policy (2026-08-17). `frank-default-large` is the Qwen3.8-27B PP boot policy (2026-08-17). `frank-large` is the Qwen3.8-27B PP
head, not the Ling path. See git history for the full decision table. head, not the Ling path. See git history for the full decision table.
@@ -5,7 +5,7 @@
# Qwen3.8-27B Speculative-Decoding Matrix # Qwen3.8-27B Speculative-Decoding Matrix
This page records the speculative-decoding benchmark history and the This page records the speculative-decoding benchmark history and the
production tuning for the Qwen3.8-27B service (`frank-default-large`), production tuning for the Qwen3.8-27B service (`frank-large`),
now a pipeline-parallel head with a Dankserver worker. now a pipeline-parallel head with a Dankserver worker.
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68) ## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
@@ -145,7 +145,7 @@ that is the long-context attention penalty, not a spec artifact.
## Production Configuration ## Production Configuration
The live tunables live in `scripts/frank-default-large.env` (head) and The live tunables live in `scripts/frank-large.env` (head) and
`scripts/frank-pp-worker.env` (worker) in this repo. The Production `scripts/frank-pp-worker.env` (worker) in this repo. The Production
state section at the top of this page records the current values: Q6_K state section at the top of this page records the current values: Q6_K
model + standalone MTP draft, split 0.68/0.32, 262144 ctx, PARALLEL=1, model + standalone MTP draft, split 0.68/0.32, 262144 ctx, PARALLEL=1,
@@ -153,6 +153,6 @@ q8_0/f16 KV.
## See Also ## See Also
- [[frank-default-large-benchmark]] - [[frank-large-benchmark]]
- [[ling-3.0-flash-trial]] - [[ling-3.0-flash-trial]]
- [[deployment]] - [[deployment]]
@@ -7,7 +7,7 @@ and the test matrix planned for 2026-08-16 morning.
Related pages: Related pages:
- [Speculative-decoding matrix](qwen3.8-spec-matrix.md) - [Speculative-decoding matrix](qwen3.8-spec-matrix.md)
- [Backend and spec benchmark](frank-default-large-benchmark.md) - [Backend and spec benchmark](frank-large-benchmark.md)
- [Dankclaw Improvements](Dankclaw%20Improvements.md) (OS-level items) - [Dankclaw Improvements](Dankclaw%20Improvements.md) (OS-level items)
- [Pipeline Parallelism: Qwen3.8 PP Deployment](Pipeline%20Parallelism/Qwen3.8%20PP%20Deployment.md) - [Pipeline Parallelism: Qwen3.8 PP Deployment](Pipeline%20Parallelism/Qwen3.8%20PP%20Deployment.md)
@@ -58,7 +58,7 @@ Consequences of the hybrid design:
## Current production state (baseline) ## Current production state (baseline)
`frank-default-large` (Dankclaw, Vulkan0, 2026-08-15): `frank-large` (Dankclaw, Vulkan0, 2026-08-15):
- Model: UD-Q8_K_XL, 31.5 GB, bundled MTP head. - Model: UD-Q8_K_XL, 31.5 GB, bundled MTP head.
- `--spec-default` (ngram-mod) with draft-mtp first in the chain. - `--spec-default` (ngram-mod) with draft-mtp first in the chain.
@@ -225,7 +225,7 @@ tensor cache warm).
| E | A with n_max 3 and 5 | n_max curve on the split | | E | A with n_max 3 and 5 | n_max curve on the split |
| F | A, plain Q8_0 model file | same-bit A/B | | F | A, plain Q8_0 model file | same-bit A/B |
| G | PP default proportional split (no `--tensor-split`) | confirms the 20/80 handicap | | G | PP default proportional split (no `--tensor-split`) | confirms the 20/80 handicap |
| Ref | production frank-default-large (Vulkan, 2 slots) | the bar to beat | | Ref | production frank-large (Vulkan, 2 slots) | the bar to beat |
Per config: `pp-test.sh bench 4` (4k prompt / 256 out) and `bench 16` Per config: `pp-test.sh bench 4` (4k prompt / 256 out) and `bench 16`
(16k / 256). Report: decode t/s (server print_timing), prefill t/s, MTP (16k / 256). Report: decode t/s (server print_timing), prefill t/s, MTP
@@ -245,5 +245,5 @@ production @16k) with the same 8-bit model.
`-dio` for RPC). `-dio` for RPC).
- llama.cpp b10433 source: `src/llama.cpp` (device selection), - llama.cpp b10433 source: `src/llama.cpp` (device selection),
`common/speculative.cpp` (MTP/ngram-mod), `ggml/src/ggml-rpc` (protocol). `common/speculative.cpp` (MTP/ngram-mod), `ggml/src/ggml-rpc` (protocol).
- Local benchmarks: [frank-default-large-benchmark](frank-default-large-benchmark.md), - Local benchmarks: [frank-large-benchmark](frank-large-benchmark.md),
[qwen3.8-spec-matrix](qwen3.8-spec-matrix.md). [qwen3.8-spec-matrix](qwen3.8-spec-matrix.md).
@@ -1,6 +1,6 @@
--- ---
type: benchmark plan type: benchmark plan
title: frank-default-large ubatch sweep title: frank-large ubatch sweep
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
tags: tags:
- llama.cpp - llama.cpp
@@ -10,9 +10,9 @@ tags:
timestamp: 2026-09-01 timestamp: 2026-09-01
--- ---
# frank-default-large ubatch sweep # frank-large ubatch sweep
This benchmark uses the active `frank-default-large` model on Dankclaw with This benchmark uses the active `frank-large` model on Dankclaw with
`llama-bench`. It does not use `llama-server`, HTTP requests, or server-log `llama-bench`. It does not use `llama-server`, HTTP requests, or server-log
scraping. Stop the production service before running it and start the service scraping. Stop the production service before running it and start the service
again afterward. again afterward.
@@ -29,9 +29,9 @@ Each point runs three 128-token repetitions by default.
Run on Dankclaw from the repository checkout: Run on Dankclaw from the repository checkout:
```bash ```bash
sudo systemctl stop frank-default-large sudo systemctl stop frank-large
./scripts/bench-ubatch.sh ./scripts/bench-ubatch.sh
sudo systemctl start frank-default-large sudo systemctl start frank-large
``` ```
The script uses the active Q6 AGENT model, an 8K prompt workload, equal The script uses the active Q6 AGENT model, an 8K prompt workload, equal
@@ -1,6 +1,6 @@
# Service Research Brief # Service Research Brief
Consolidated research from the `frank-default-small` research subagents (2026-08-14) plus the pi agent knowledge base. This is the source material behind the per-service docs pages. Consolidated research from the `frank-small` research subagents (2026-08-14) plus the pi agent knowledge base. This is the source material behind the per-service docs pages.
## Method ## Method
+19 -15
View File
@@ -51,8 +51,9 @@ covers both workloads).
- [[readarr]] — ebook/audiobook management - [[readarr]] — ebook/audiobook management
- [[prowlarr]] — indexer management - [[prowlarr]] — indexer management
- [[qbittorrent]] — torrent download client - [[qbittorrent]] — torrent download client
- [[Plex]] — native systemd media server on Dankserver (not a container) - [[Plex]] — media server on Dankserver (lscr.io/linuxserver/plex container via
- [[Tautulli]] — Plex statistics and notifications, snap-packaged quadlet; CPU decode only — no GPU passthrough)
- [[Tautulli]] — Plex statistics and notifications (v2.18.1-ls242, quadlet unit)
- [[mariadb]] — RomM database backend - [[mariadb]] — RomM database backend
- [[redis]] — RomM cache/queue backend - [[redis]] — RomM cache/queue backend
@@ -62,7 +63,7 @@ covers both workloads).
- [[SearXNG Providers]] — unblocking Google & other upstream engines behind the VPN - [[SearXNG Providers]] — unblocking Google & other upstream engines behind the VPN
- [[Gluetun]] — the VPN tunnel container - [[Gluetun]] — the VPN tunnel container
- [[Valkey]] — SearXNG cache and limiter store - [[Valkey]] — SearXNG cache and limiter store
- [[ComfyUI]] — node-based AI image generation (CPU) - [[ComfyUI]] — node-based AI image generation (ROCm GPU, gfx1151 build)
- [[OpenWebUI]] — chat UI for local LLMs - [[OpenWebUI]] — chat UI for local LLMs
- [[Hermes]] — the Nous Research agent - [[Hermes]] — the Nous Research agent
- [[Hermes-Dashboard]] — the Hermes web management UI (config, keys, sessions) - [[Hermes-Dashboard]] — the Hermes web management UI (config, keys, sessions)
@@ -112,35 +113,38 @@ The LLM inference fleet on the two hosts. Full index:
- [[deployment]] — unified deploy and update process for both servers - [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[freetoken-frank-default-small]] — the active frank-default-small runbook: FreeToken v0.1.2 serving Qwen3.6-35B-A3B-FP8 on Dankserver (2026-09-02) - [[freetoken-frank-small]] — the active frank-small runbook: FreeToken v0.1.2 serving Qwen3.6-35B-A3B-FP8 on Dankserver (2026-09-02)
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090 - [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (restored 2026-09-01) - [[kingjones-strix-mtp]] — the Qwen3.8-27B-Q6_0 ROCmFPX AGENT era with the
- [[agentionai-fast-imatrix]] — the ACTIVE frank-default-large runbook: standalone MTP draft (Dankclaw, 2026-08-25 → 2026-08-29; historical)
- [[agentionai-fast-imatrix]] — the retired frank-large runbook:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB, agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
Q4_0_ROCMFP4_FAST) + own MTP draft head, build-qwen38 (2026-08-29) Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
- [[kingjones-strix-rocmfp4]] — retired frank-default-large runbook: rollback record)
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B params, 113.47 GiB) - [[kingjones-strix-rocmfp4]] — the ACTIVE frank-large runbook since
2026-08-30: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B params,
113.47 GiB) on the card's pinned kingjones30/ROCmFPX build
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure - [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw - [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24) - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active) - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02) - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for frank-default-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26) - [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for frank-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22) - [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21) - [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the RTX 3090 box hardware - [[Dankserver System Specs]] — the RTX 3090 box hardware
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware - [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large)
- [[Dankclaw Improvements]] — host-level changes on the CIRU box - [[Dankclaw Improvements]] — host-level changes on the Strix Halo box
- [[frank-default-sightsound]] — ACTIVE FLM NPU sight+sound runbook on Dankclaw: - [[frank-npu]] — ACTIVE FLM NPU sight+sound runbook on Dankclaw:
Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 7771; whole local Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 7771; whole local
registry served on one port with per-request model routing; includes the registry served on one port with per-request model routing; includes the
IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash
### Benchmarks and Sizing ### Benchmarks and Sizing
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head - [[frank-large-benchmark]] — backend/spec benchmark behind the old head
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results - [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
- [[frank-default-small-benchmark]] — the small-model benchmark suite - [[frank-small-benchmark]] — the small-model benchmark suite
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes - [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU - [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes - [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
+48 -13
View File
@@ -11,6 +11,41 @@ timestamp: 2026-09-14
# Bundle Change Log # Bundle Change Log
## 2026-09-03 — full fact-check sweep of the docs tree (docs-only, deploy no-op)
- Scope: every page under `/serverAppConfigs-docs/` re-verified against live
hosts (read-only SSH), quadlet unit files, and the Caddyfile; surgical edits
only — no pages created, renamed, or deleted. Per the deploy planner, a
docs-only change is a deploy no-op.
- Dankserver core rewritten to live quadlet facts: the *arr stack + qBittorrent
+ gluetun run on **FrankBook-Pro** (units installed to
`/home/core/.config/containers/systemd/` via the frankbook-pro deploy
handler; `PUID=0/PGID=0`, data roots `/Volumes/COMPLETE/<svc>-data:/config`;
qB pinned `4.6.7` sharing gluetun's netns with 8080 + 6881 published on the
gluetun unit; readarr pinned `amd64-1.0.980`; no vhosts — direct IP:port,
exempt by construction from the guard chain). Jellyfin, Plex, RomM (8085→8080
guarded), Calibre-Web (`0.6.27-ls397` + universal-calibre mod; no standalone
calibre-server), mariadb/redis/Tautulli/Entropy verified against live units.
- Caddy guard chain documented to the live mechanism end-to-end: Anubis :8923 →
internal :8016 → post-auth :8017 routing; guarded vhosts (search, grafana,
prometheus, loki, seerr) route post-auth to container-name upstreams;
exemptions named by construction reason (auth/IdP itself, gitea smart-HTTP,
ai→:3005 own auth, hermes/hermesadmin internal-only via DNS absence, pihole
basicauth edge → :8083).
- `Gitea.md`: data roots + secrets corrected to the live unit paths
(`/home/dank/Containers/gitea/{data,postgres,env}`); UID gotcha corrected —
the units carry no `User=`/UserNS override, so container uid 1000 is literally
host uid 1000 (no subuid offset for these containers).
- Dedupe: `Prowlarr.md`/`prowlarr.md` collapsed into one canonical lowercase
page via rename (git history retains both versions; index link unchanged).
- LLM docs: archive pages (`DFlash DSpark Speculative Decoding`,
`ling-3.0-flash-trial`) reworded from current-tense to dated historical
framing with pointers to the active runbook [[kingjones-strix-rocmfp4]];
LLM `index.md` + `deployment.md` fleet tables and health-gate text updated to
the 2026-08-30 fleet state (STRIX on the pinned ROCmFPX fork; Huihui
abliterated FP8 on FreeToken; sightsound = FLM v1.0.1 on XDNA2, port 7771,
health `/v1/version` — FLM has no `/health` route).
- Suite green after the sweep: `./tests/run.sh` → 233 pass / 0 fail, including
the docs-completeness seam that requires every page to stay index-linked.
## 2026-09-01 (late) — FrankBook-Pro stack completed: tunnel live, qB wired into the arrs ## 2026-09-01 (late) — FrankBook-Pro stack completed: tunnel live, qB wired into the arrs
- WireGuard credentials provisioned (second Proton config, server `PA-12`, - WireGuard credentials provisioned (second Proton config, server `PA-12`,
endpoint `51.179.177.2:51820`) → `/home/core/Containers/gluetun/env` endpoint `51.179.177.2:51820`) → `/home/core/Containers/gluetun/env`
@@ -95,9 +130,9 @@ timestamp: 2026-09-14
rules), [[Jellyfin]] gained the LDAP-Auth plugin section. Docs-only rules), [[Jellyfin]] gained the LDAP-Auth plugin section. Docs-only
commit → deploy no-op. commit → deploy no-op.
## 2026-09-01 — frank-default-small: two compressed-tensors FP8 swaps fail to load; Huihui restored ## 2026-09-01 — frank-small: two compressed-tensors FP8 swaps fail to load; Huihui restored
- `/scripts/frank-default-small.env`, `/scripts/serve-frank-small.sh`, - `/scripts/frank-small.env`, `/scripts/serve-frank-small.sh`,
`/scripts/frank-default-small.service` (`af6cd72` → `34ab6b4` → reverted `/scripts/frank-small.service` (`af6cd72` → `34ab6b4` → reverted
in `4d5bea4`) — attempted model swaps on FreeToken v0.1.2, both in `4d5bea4`) — attempted model swaps on FreeToken v0.1.2, both
live-verified as loader failures and reverted the same day to the Huihui live-verified as loader failures and reverted the same day to the Huihui
FP8 checkpoint: FP8 checkpoint:
@@ -114,7 +149,7 @@ timestamp: 2026-09-14
- Both failed cleanly at load (37 GB fits RAM — no swap-thrash, no - Both failed cleanly at load (37 GB fits RAM — no swap-thrash, no
outage); service was back on Huihui within the hour and re-verified with outage); service was back on Huihui within the hour and re-verified with
a real completion (reasoning parses; `17*23 → 391`, natural stop). a real completion (reasoning parses; `17*23 → 391`, natural stop).
- `/serverAppConfigs-docs/LLM/freetoken-frank-default-small.md` — quant - `/serverAppConfigs-docs/LLM/freetoken-frank-small.md` — quant
matrix updated: modelopt block-fp8 is the ONLY FP8 path that loads; both matrix updated: modelopt block-fp8 is the ONLY FP8 path that loads; both
compressed-tensors shapes documented with their exact errors. compressed-tensors shapes documented with their exact errors.
- **Deploy health-check trap (second occurrence)**: `deploy.sh` reported - **Deploy health-check trap (second occurrence)**: `deploy.sh` reported
@@ -164,8 +199,8 @@ timestamp: 2026-09-14
+ the two ownership regimes; [[Deployment Flow]] service-manager table + the two ownership regimes; [[Deployment Flow]] service-manager table
corrected (dankserver media is no longer "bare containers, inventory only"). corrected (dankserver media is no longer "bare containers, inventory only").
## 2026-08-30 (night II) — frank-default-sightsound deployed; NPU SVA/IOMMU root cause ## 2026-08-30 (night II) — frank-npu deployed; NPU SVA/IOMMU root cause
- **Deployed** `frank-default-sightsound` on Dankclaw: FLM v1.0.1 (`/usr/bin/flm`, - **Deployed** `frank-npu` on Dankclaw: FLM v1.0.1 (`/usr/bin/flm`,
dpkg `fastflowlm`) serving the whole local registry (36 tags) on port 52626; dpkg `fastflowlm`) serving the whole local registry (36 tags) on port 52626;
primary model Gemma4 E4B IT NPU2 (vision + audio towers). First deploy primary model Gemma4 E4B IT NPU2 (vision + audio towers). First deploy
crash-looped every 10 s: `Open /dev/accel/accel0 failed (err=-19)`. crash-looped every 10 s: `Open /dev/accel/accel0 failed (err=-19)`.
@@ -177,7 +212,7 @@ timestamp: 2026-09-14
kernel line — a consequence, not a GPU fault. kernel line — a consequence, not a GPU fault.
- **Fix**: `iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps - **Fix**: `iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps
the iGPU default domain identity-mapped — zero DMA-translation overhead for the iGPU default domain identity-mapped — zero DMA-translation overhead for
frank-default-large — while enabling IOMMU hardware + PASID, which SVA needs; frank-large — while enabling IOMMU hardware + PASID, which SVA needs;
PASID domains are paged regardless of the default-domain type). Post-boot: PASID domains are paged regardless of the default-domain type). Post-boot:
30 IOMMU groups, zero SVA-bind errors since boot, unit `enabled`, 30 IOMMU groups, zero SVA-bind errors since boot, unit `enabled`,
`NRestarts=0` since boot. `NRestarts=0` since boot.
@@ -187,18 +222,18 @@ timestamp: 2026-09-14
279 prompt tokens consumed); audio tower proven with a control pair (440 Hz 279 prompt tokens consumed); audio tower proven with a control pair (440 Hz
tone via `input_audio` = 37 prompt tokens vs 15 for the identical text-only tone via `input_audio` = 37 prompt tokens vs 15 for the identical text-only
control). Health probe is `/v1/version` — FLM has no `/health` route. control). Health probe is `/v1/version` — FLM has no `/health` route.
- `/scripts/frank-default-sightsound.env`, `/scripts/serve-frank-default-sightsound.sh`, - `/scripts/frank-npu.env`, `/scripts/serve-frank-npu.sh`,
`/scripts/frank-default-sightsound.service` + deployer allowlist entries `/scripts/frank-npu.service` + deployer allowlist entries
(`c3ed3a6`, `cf0c541`) — service files and deploy wiring; unit is boot-persistent (`c3ed3a6`, `cf0c541`) — service files and deploy wiring; unit is boot-persistent
(`WantedBy=multi-user.target`, enabled, `Restart=on-failure`, `RestartSec=10`, (`WantedBy=multi-user.target`, enabled, `Restart=on-failure`, `RestartSec=10`,
`LimitMEMLOCK=infinity` — FLM requires unlimited memlock). `LimitMEMLOCK=infinity` — FLM requires unlimited memlock).
- **Docs**: new runbook [[frank-default-sightsound]] (linked from both indexes), - **Docs**: new runbook [[frank-npu]] (linked from both indexes),
AGENTS.md fleet table + runtime section updated with the IOMMU/SVA hard AGENTS.md fleet table + runtime section updated with the IOMMU/SVA hard
requirement. Client wiring: pi `models.json` registers the endpoint under the requirement. Client wiring: pi `models.json` registers the endpoint under the
Dankclaw provider with the exact FLM registry tag as model id. Dankclaw provider with the exact FLM registry tag as model id.
## 2026-08-30 (night) — frank fleet double swap: STRIX on its pinned fork, Huihui on FreeToken ## 2026-08-30 (night) — frank fleet double swap: STRIX on its pinned fork, Huihui on FreeToken
- **Dankclaw / frank-default-large**: swapped to - **Dankclaw / frank-large**: swapped to
`kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF` on the card's `kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF` on the card's
pinned `kingjones30/ROCmFPX` runtime (built at pinned `kingjones30/ROCmFPX` runtime (built at
`/home/dank/GitPackages/kingjones-ROCmFPX/build`, `LLAMA_BUILD_WEBUI=OFF` `/home/dank/GitPackages/kingjones-ROCmFPX/build`, `LLAMA_BUILD_WEBUI=OFF`
@@ -212,7 +247,7 @@ timestamp: 2026-09-14
git flow; manual-process race during validation caught and fixed git flow; manual-process race during validation caught and fixed
(unit was crash-looping on a bound port while its health check (unit was crash-looping on a bound port while its health check
passed — check MainPID, not just /health). passed — check MainPID, not just /health).
- **Dankserver / frank-default-small**: swapped the model within - **Dankserver / frank-small**: swapped the model within
FreeToken v0.1.2 to `batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8` FreeToken v0.1.2 to `batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8`
(native block-fp8, same loader path and pool plan; 35 GB downloaded (native block-fp8, same loader path and pool plan; 35 GB downloaded
via `hf`, official checkpoint retained for rollback). Learned: via `hf`, official checkpoint retained for rollback). Learned:
@@ -220,7 +255,7 @@ timestamp: 2026-09-14
tok/s verified post-deploy. Both `.service` Descriptions refreshed. tok/s verified post-deploy. Both `.service` Descriptions refreshed.
- Docs: [[kingjones-strix-rocmfp4]] reactivated as the large runbook - Docs: [[kingjones-strix-rocmfp4]] reactivated as the large runbook
(runtime build + swap procedure + validation), [[agentionai-fast-imatrix]] (runtime build + swap procedure + validation), [[agentionai-fast-imatrix]]
marked RETIRED (rollback record), [[freetoken-frank-default-small]] marked RETIRED (rollback record), [[freetoken-frank-small]]
model sections updated, LLM index + AGENTS fleet tables refreshed. model sections updated, LLM index + AGENTS fleet tables refreshed.
## 2026-08-30 (late) — download chain proven end to end; qB credentials persisted ## 2026-08-30 (late) — download chain proven end to end; qB credentials persisted
+2 -2
View File
@@ -18,8 +18,8 @@
{"name":"Caddy","category":"admin","description":"The reverse proxy.","url":"","docs":"","probe":"127.0.0.1:80","method":"http","wip":false,"showLink":false}, {"name":"Caddy","category":"admin","description":"The reverse proxy.","url":"","docs":"","probe":"127.0.0.1:80","method":"http","wip":false,"showLink":false},
{"name":"dnsmasq","category":"admin","description":"LAN name resolution.","url":"","docs":"","probe":"127.0.0.1:53","method":"tcp","wip":false,"showLink":false}, {"name":"dnsmasq","category":"admin","description":"LAN name resolution.","url":"","docs":"","probe":"127.0.0.1:53","method":"tcp","wip":false,"showLink":false},
{"name":"Gluetun","category":"admin","description":"The outbound VPN tunnel.","url":"","docs":"","probe":"127.0.0.1:51820","method":"tcp","wip":true,"showLink":false}, {"name":"Gluetun","category":"admin","description":"The outbound VPN tunnel.","url":"","docs":"","probe":"127.0.0.1:51820","method":"tcp","wip":true,"showLink":false},
{"name":"frank-default-large","category":"admin","description":"The large LLM (27B) on Dankclaw.","url":"","docs":"","probe":"127.0.0.1:7770","method":"http","wip":false,"showLink":false}, {"name":"frank-large","category":"admin","description":"The large LLM (27B) on Dankclaw.","url":"","docs":"","probe":"127.0.0.1:7770","method":"http","wip":false,"showLink":false},
{"name":"frank-default-small","category":"admin","description":"The small LLM (35B MoE) on Dankserver.","url":"","docs":"","probe":"192.168.0.235:7770","method":"http","wip":false,"showLink":false}, {"name":"frank-small","category":"admin","description":"The small LLM (35B MoE) on Dankserver.","url":"","docs":"","probe":"192.168.0.235:7770","method":"http","wip":false,"showLink":false},
{"name":"mariadb","category":"admin","description":"The media-stack database.","url":"","docs":"","probe":"192.168.0.235:3306","method":"tcp","wip":false,"showLink":false}, {"name":"mariadb","category":"admin","description":"The media-stack database.","url":"","docs":"","probe":"192.168.0.235:3306","method":"tcp","wip":false,"showLink":false},
{"name":"valkey","category":"admin","description":"The SearXNG cache.","url":"","docs":"","probe":"127.0.0.1:6379","method":"tcp","wip":false,"showLink":false}, {"name":"valkey","category":"admin","description":"The SearXNG cache.","url":"","docs":"","probe":"127.0.0.1:6379","method":"tcp","wip":false,"showLink":false},
{"name":"Postgres","category":"admin","description":"The Gitea database.","url":"","docs":"","probe":"127.0.0.1:5432","method":"tcp","wip":false,"showLink":false}, {"name":"Postgres","category":"admin","description":"The Gitea database.","url":"","docs":"","probe":"127.0.0.1:5432","method":"tcp","wip":false,"showLink":false},