serverAppConfigs/serverAppConfigs-docs/index.md

# Server App Configs

Podman deployment configuration for two homelab servers: Dankserver and Dankclaw.

This repository is the single source of truth for both hosts. It tracks the
container stacks and the LLM inference fleet. Edits flow through this repo,
then deploy to the targets with `scripts/deploy.sh` (one script per target
covers both workloads).

> 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
> into this repo with its full git history. Its knowledge base now lives at
> [serverAppConfigs-docs/LLM/index.md](LLM/index.md).

## Hosts

| Host | IP | Role | OS |
|------|----|------|----|
| Dankserver | 192.168.0.235 | Media & library stack | Ubuntu 24.04 |
| Dankclaw | 192.168.0.143 | AI, search & development stack | Ubuntu 26.04 |

## Stacks

- [[Dankserver Stack]] — RomM, Copyparty, Calibre, Calibre-Web, Caddy
- [[Dankclaw Stack]] — SearXNG (via Gluetun), ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Caddy

## Architecture

- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
- [[Storage Policy]] — bind mounts, data locations, ownership
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
- [[Testing Policy]] — the test seams and how to run them
- [[E2E Suite]] — the Playwright suite that drives the live guard chain
- [[Container Network Namespaces]] — which local services are addressable from each Podman network namespace on Dankclaw

## Dankserver Storage

- [[Entropy Volume]] — command-line unlock and mount helper

## Dankserver Services

- [[RomM]] — retro game manager
- [[RomM-Scraping]] — metadata providers, cover art, naming conventions
- [[Copyparty]] — file sharing and streaming
- [[Jellyfin]] — media server (movies, TV, music)
- [[Calibre]] — e-book library engine
- [[Calibre-Web]] — e-book web frontend
- [[ARR-Stack]] — automated media download overview
- [[sonarr]] — TV show management
- [[radarr]] — movie management
- [[lidarr]] — music management
- [[readarr]] — ebook/audiobook management
- [[prowlarr]] — indexer management
- [[qbittorrent]] — torrent download client
- [[Plex]] — media server on Dankserver (lscr.io/linuxserver/plex container via
  quadlet; CPU decode only — no GPU passthrough)
- [[Tautulli]] — Plex statistics and notifications (v2.18.1-ls242, quadlet unit)
- [[mariadb]] — RomM database backend
- [[redis]] — RomM cache/queue backend

## Dankclaw Services

- [[SearXNG]] — privacy metasearch (VPN-routed)
- [[SearXNG Providers]] — unblocking Google & other upstream engines behind the VPN
- [[Gluetun]] — the VPN tunnel container
- [[Valkey]] — SearXNG cache and limiter store
- [[ComfyUI]] — node-based AI image generation (ROCm GPU, gfx1151 build)
- [[OpenWebUI]] — chat UI for local LLMs
- [[Hermes]] — the Nous Research agent
- [[Hermes-Dashboard]] — the Hermes web management UI (config, keys, sessions)
- [[Hermes-WebUI]] — the Hermex server (loopback API for the iPhone app, SSH-tunneled)
- [[Gitea]] — Git hosting (migrated)
- [[Postgres]] — Gitea database backend
- [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
- [[Marreta]] — content-cleaning proxy (guard-chained; shares SearXNG's Gluetun egress)

## Infrastructure

- [[LAN-DNS]] — name resolution, Pi-hole, router DNS hand-off
- [[Pi-hole]] — the LAN DNS server (quadlet, records, web UI)
- [[Caddy]] — reverse proxy and TLS
- [[Anubis]] — bot challenge proxy fronting the guarded vhosts (chain, policy, gotchas)
- [[Authentik]] — the homelab's SSO / OIDC provider (forward_auth)
- [[authentik-postgres]] — Authentik database service
- [[authentik-server]] — Authentik server + Caddy outpost service
- [[authentik-worker]] — Authentik background worker service
- [[Homepage]] — password-protected service index
- [[Homepage Design Grilling]] — homepage decision log and open questions
- [[Observability]] — Grafana, Prometheus, Loki, Alloy, node-exporter, Alertmanager: the stack, its quadlet deployment, its exposure, and what is still missing
- [[Grafana]] — dashboards (datasource and the LLM fleet dashboard provisioned as code since 2026-09-16)
- [[Prometheus]] — metrics collection (5 alert rules + 1 recording rule, delivered via Alertmanager since 2026-09-17)
- [[Alertmanager]] — alert grouping and Signal delivery (webhook → host-side relay → signal-cli)
- [[Node-Exporter]] — host metrics
- [[Loki]] — log storage
- [[Alloy]] — log shipping (replaces Promtail)
- [[FrankBook-Pro]] — Mac mini *arr + qBittorrent inside podman machine (gluetun-gated)

## Research

- [[Service Research Brief]] — consolidated subagent research behind these pages
- [[ComfyUI ROCm and Vulkan]] — live-verified GPU path for ComfyUI on Strix Halo
- [[Lemonade and sd.cpp]] — the ggml-family image-generation alternative
- [[Llama Cpp Runtimes]] — mainline vs charlie12345/ROCmFPX vs ik_llama.cpp: quantization + server fit
- [[Llama Cpp Code Diff]] — code-level quantization diff of the three llama.cpp runtimes
- [[SGLang]] — the structured-generation serving engine (llama.cpp/vLLM comparison + our fit)
- [[Backblaze Integration]] — off-site backup plan (future)
- [[Jellyfin]] — media server research brief
- [[ARR-Stack]] — automated download stack research brief

## LLM Fleet

The LLM inference fleet on the two hosts. Full index:
[serverAppConfigs-docs/LLM/index.md](LLM/index.md).

### Deployment and Topology

- [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook:
  vllm-radiance as ONE tensor-parallel process (TP=2 over the R4D one-shot
  allreduce) serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq (MXFP4
  W4A8, DFlash2 drafter, unquantized bf16 KV since 2026-09-21) on Dankserver's
  2x Radeon AI PRO R9700 behind the :7770 endpoint — the quantization-aware
  entrypoint, the three-arm battery that chose TP=2, verified capabilities,
  and the record of the 2026-09-16 two-replica era
- [[frank-medium-model-swap]] — how to swap the checkpoint behind frank-medium:
  the streamlining toolchain (hf-manifest.py, hf-fetch.sh,
  swap-frank-medium.sh), the quantization-aware entrypoint contract, the
  quadlet edit, the deploy, the capability battery, and the rollback path
- [[frank-medium-llamacpp-dflash2]] — RETIRED 2026-09-16 → 2026-09-20:
  mainline llama.cpp (gfx1201 HIP build) serving orcarouter Qwen3.8-27B at
  Q8_0 with the DFlash2 drafter as ONE layer-split process (32:33, two
  163,840-token slots); holds the batch/ubatch optimum curve and the
  per-layer split arithmetic the rollback recipe points back to
- [[Dual-R9700-Cooperation]] — whether the two R9700s can run ONE model: the
  RCCL/PCIe-atomics evidence on the old TUF GAMING B650-E (which closed tensor
  parallelism too), what the 2026-09-20 ProArt B850-Creator x8/x8 swap changed
  — both chains `Routing+`, both endpoints `ReqEn+` — and the measured
  three-way verdict: radiance TP=2 won and is the deployed shape
- [[frank-medium-rocmfpx]] — PRIOR frank-medium deployment (llama.cpp era):
  charlie12345/ROCmFPX serving kingjones777/
  Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the runtime it
  replaced
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep (bs1/bs3/bs4/bs6, 8k–700k KV) on the retired Dankserver RTX 3090
- [[kingjones-strix-mtp]] — the Qwen3.8-27B-Q6_0 ROCmFPX AGENT era with the
  standalone MTP draft (Dankclaw, 2026-08-25 → 2026-08-29; historical)
- [[agentionai-fast-imatrix]] — the retired frank-large runbook:
  agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
  Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
  rollback record)
- [[frank-large-unsloth-qwen38-27b-q8xl]] — the ACTIVE frank-large runbook
  since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
  (unsloth b11030, ROCm gfx1151) serving
  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the z-lab
  DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one 262,144-token
  conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp v22.4.0
  template, the 114 GiB GTT memory model, client contract
- [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (replaced by
  [[frank-large-unsloth-qwen38-27b-q8xl]]; same-day swap): the Kairic Edge
  runtime (ciru-ai/ROCmFPX kairic-edge-qwen38-27b-v1.2, TheRock 7.15.0,
  Composable Kernel) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (a 27B
  dense model, main GGUF + FFN/GDN/GDN-Output .pfs sidecars) on the gfx1151
  iGPU — one 262,144-token conversation, native MTP depth 4 + n-gram
  drafting, compatibility mode, the 114 GiB GTT memory model, client contract
- [[frank-large-unsloth-qwen38-flash-next]] — RETIRED 2026-09-21 (replaced by
  [[frank-large-kairic-edge-qwen38-27b]]; same-day swap): the unsloth
  llama.cpp prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving
  unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (a 125B MoE / 6B active) on the
  gfx1151 iGPU — one 200,000-token conversation, MTP depth 2 via the
  explicit -md shared-Q8_0 head, the 114 GiB GTT memory model, client contract
- [[frank-large-ciru-orca]] — RETIRED 2026-09-21 (replaced by
  [[frank-large-unsloth-qwen38-flash-next]]; rollback target, 2026-09-19 →
  09-21): the CIRU v4.4.1 runtime (private host build, QSA hotfix) serving
  jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151 iGPU — one
  262,144-token conversation, MTP depth 4, 4096 MiB PLE cache, runtime-root
  gate, client contract
- [[halogen-flash-server]] — RETIRED 2026-09-19 (replaced by
  [[frank-large-ciru-orca]]; rollback target, 2026-09-16 → 09-19):
  peonist-ai/halogen-flash-server 0.5.8 serving peonist-ai/halogen-qwen3.8-
  flash-next (w4b `.hgn` + quality overlay) on the gfx1151 iGPU — two 200k
  conversations over one 400,384-position pool, deployed flags, memory model,
  client contract
- [[frank-large-ik-uncensored]] — the RETIRED 2026-09-16 frank-large runbook
  and rollback target (2026-09-15 → 09-16): ik_llama.cpp serving
  mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 +
  Strix Halo hybrid — its measured envelope is why the fleet went back
- [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
  `amd_iommu=off` since 2026-09-11 (13-16 % prefill), what it cost
  (the NPU endpoint), and how to revert
- [[kingjones-strix-rocmfp4]] — the PRIOR frank-large runbook
  (2026-08-30 → 2026-09-11) and the rollback target:
  kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF on the card's
  pinned kingjones30/ROCmFPX build
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[frank-small-gemma4-12b]] — the ACTIVE frank-small runbook since
  2026-09-17: mainline llama.cpp serving
  HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M
  (MTP assistant sidecar + ngram-mod, q8_0 KV, --fit → 262k) on
  Dankclaw's RTX 3090, port 7771
- [[frank-small-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
  rollback target): vLLM serving google/gemma-4-12B-it-qat-w4a16-ct (w4a16
  Marlin) with the bf16 MTP assistant on the RTX 3090 — bf16 KV, 289,647-
  token pool, 2 concurrent full-context sessions, native text+image+audio
- [[frank-small-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target):
  the prior frank-small runbook (2026-09-16 → 09-17), ISTA-DASLab
  Qwen3.8-27B-GSQ-RCO IQ3_S with own in-file MTP head + BF16 projector
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for the old frank-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
- [[freetoken-frank-small]] — RETIRED 2026-09-15: the FreeToken/CUDA frank-small runbook, retired with the RTX 3090
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the 2x R9700 box hardware (frank-medium)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large)
- [[Dankclaw Improvements]] — host-level changes on the Strix Halo box
- [[Swap Configuration]] — the 64 GiB `/swap.img` host swapfile (raised from
  8 GiB 2026-09-12): fstab-driven `swap.img.swap` unit, the safe resize
  ordering that drains the old file into fresh swap instead of into a host
  whose `MemAvailable` lies by the size of the model, and the swappiness caveat
- [[frank-npu]] — the RETIRED FLM NPU endpoint on Dankclaw (2026-09-11):
  Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 7771; kept as the
  retirement record, including the IOMMU/SVA err=-19 root cause and
  where every dependent route moved

### Benchmarks and Sizing

- [[frank-large-benchmark]] — backend/spec benchmark behind the old head
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
- [[frank-small-benchmark]] — the small-model benchmark suite
- [[frank-medium-batch-ubatch]] — the 2026-09-16 batch/ubatch sweep on the 2x
  R9700 layer split: prefill PEAKS at ubatch 512 and falls on both sides, the
  logical batch buys nothing measurable, decode never moves, and the ubatch's
  uneven VRAM give-back (1.30 GiB on card 0 against 2.41 GiB on card 1) moved
  the tensor split from 36:29 to 35:30
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
- [[x4-pair-experiment]] — what a 3090 + Strix Halo combination can and cannot
  do in mainline llama.cpp (measured 2026-09-16): the tensor inventory that
  decides a split, the one flag (`-ot`) that kills per-tensor placement across
  devices, the two-device layer splits that DO work (pure-Vulkan and cross-vendor
  CUDA+Vulkan, +7 % decode at 64k), and the harness-parsing trap that made the
  first round of measurements say otherwise

### Memory and Hardware

- [[RAM CPU Offloading in VLLM]] — KV cache swap between GPU VRAM and system RAM
- [[Hybrid CPU GPU Processing Dankserver]] — NVIDIA GPU compute with system RAM offload
- [[Shared Memory Architecture Dankclaw]] — AMD unified memory and ROCm configuration
- [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash
- [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU

### Retired Topology and History

- [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP)
- [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint
- [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting
- [[ling-3.0-flash-trial]] — the Ling-3.0-Flash trial runbook
- [[ling-3.0-flash-vllm-plan]] — the Ling-3.0-Flash vLLM plan (retired)

### vLLM Reference (historical)

- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[vllm_research_findings]] — research findings from the vLLM era

### Pipeline Parallelism (archive)

- [[Ethernet Bandwidth Math]] — the bandwidth budget for the PP pair
- [[Expected Performance]] — expected PP throughput
- [[llama.cpp RPC]] — the RPC transport between head and worker
- [[Mixed-Vendor Experiments]] — cross-vendor PP experiments
- [[Qwen3.6 27B Spec Methods]] — spec-decoding methods for the 27B
- [[Sample Script Set]] — reference scripts for the PP pair
- [[vLLM Pipeline Parallelism]] — vLLM's PP implementation

## Update Log

- [[log]] — chronological record of knowledge-base changes

## See Also

- [AGENTS.md](../AGENTS.md) — repository rules
- [serverAppConfigs-docs/LLM/index.md](LLM/index.md) — the LLM fleet knowledge base
- [dankware Configuration Knowledgebase](~/.pi/agent/docs/index.md) — the "root" level system knowlegebase