188 lines
9.3 KiB
Markdown
188 lines
9.3 KiB
Markdown
# Server App Configs
|
||
|
||
Podman deployment configuration for two homelab servers: Dankserver and Dankclaw.
|
||
|
||
This repository is the single source of truth for both hosts. It tracks the
|
||
container stacks and the LLM inference fleet. Edits flow through this repo,
|
||
then deploy to the targets with `scripts/deploy.sh` (one script per target
|
||
covers both workloads).
|
||
|
||
> 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
|
||
> into this repo with its full git history. Its knowledge base now lives at
|
||
> [serverAppConfigs-docs/LLM/index.md](LLM/index.md).
|
||
|
||
## Hosts
|
||
|
||
| Host | IP | Role | OS |
|
||
|------|----|------|----|
|
||
| Dankserver | 192.168.0.235 | Media & library stack | Ubuntu 24.04 |
|
||
| Dankclaw | 192.168.0.143 | AI, search & development stack | Ubuntu 26.04 |
|
||
|
||
## Stacks
|
||
|
||
- [[Dankserver Stack]] — RomM, Copyparty, Calibre, Calibre-Web, Caddy
|
||
- [[Dankclaw Stack]] — SearXNG (via Gluetun), ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Caddy
|
||
|
||
## Architecture
|
||
|
||
- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
|
||
- [[Storage Policy]] — bind mounts, data locations, ownership
|
||
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
|
||
- [[Testing Policy]] — the test seams and how to run them
|
||
|
||
## Dankserver Storage
|
||
|
||
- [[Entropy Volume]] — command-line unlock and mount helper
|
||
|
||
## Dankserver Services
|
||
|
||
- [[RomM]] — retro game manager
|
||
- [[RomM-Scraping]] — metadata providers, cover art, naming conventions
|
||
- [[Copyparty]] — file sharing and streaming
|
||
- [[Jellyfin]] — media server (movies, TV, music)
|
||
- [[Calibre]] — e-book library engine
|
||
- [[Calibre-Web]] — e-book web frontend
|
||
- [[ARR-Stack]] — automated media download overview
|
||
- [[sonarr]] — TV show management
|
||
- [[radarr]] — movie management
|
||
- [[lidarr]] — music management
|
||
- [[readarr]] — ebook/audiobook management
|
||
- [[prowlarr]] — indexer management
|
||
- [[qbittorrent]] — torrent download client
|
||
- [[Plex]] — native systemd media server on Dankserver (not a container)
|
||
- [[Tautulli]] — Plex statistics and notifications, snap-packaged
|
||
- [[mariadb]] — RomM database backend
|
||
- [[redis]] — RomM cache/queue backend
|
||
|
||
## Dankclaw Services
|
||
|
||
- [[SearXNG]] — privacy metasearch (VPN-routed)
|
||
- [[SearXNG Providers]] — unblocking Google & other upstream engines behind the VPN
|
||
- [[Gluetun]] — the VPN tunnel container
|
||
- [[Valkey]] — SearXNG cache and limiter store
|
||
- [[ComfyUI]] — node-based AI image generation (CPU)
|
||
- [[OpenWebUI]] — chat UI for local LLMs
|
||
- [[Hermes]] — the Nous Research agent
|
||
- [[Hermes-Dashboard]] — the Hermes web management UI (config, keys, sessions)
|
||
- [[Hermes-WebUI]] — the Hermex server (loopback API for the iPhone app, SSH-tunneled)
|
||
- [[Gitea]] — Git hosting (migrated)
|
||
- [[Postgres]] — Gitea database backend
|
||
- [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
|
||
|
||
## Infrastructure
|
||
|
||
- [[LAN-DNS]] — name resolution, Pi-hole, router DNS hand-off
|
||
- [[Pi-hole]] — the LAN DNS server (quadlet, records, web UI)
|
||
- [[Caddy]] — reverse proxy and TLS
|
||
- [[Anubis]] — bot challenge proxy fronting the guarded vhosts (chain, policy, gotchas)
|
||
- [[Authentik]] — the homelab's SSO / OIDC provider (forward_auth)
|
||
- [[authentik-postgres]] — Authentik database service
|
||
- [[authentik-server]] — Authentik server + Caddy outpost service
|
||
- [[authentik-worker]] — Authentik background worker service
|
||
- [[Homepage]] — password-protected service index
|
||
- [[Homepage Design Grilling]] — homepage decision log and open questions
|
||
- [[Observability]] — Grafana, Prometheus, Loki, Alloy
|
||
- [[Grafana]] — dashboards
|
||
- [[Prometheus]] — metrics collection
|
||
- [[Node-Exporter]] — host metrics
|
||
- [[Loki]] — log storage
|
||
- [[Alloy]] — log shipping (replaces Promtail)
|
||
- [[FrankBook-Pro]] — Mac mini *arr + qBittorrent inside podman machine (gluetun-gated)
|
||
|
||
## Research
|
||
|
||
- [[Service Research Brief]] — consolidated subagent research behind these pages
|
||
- [[ComfyUI ROCm and Vulkan]] — live-verified GPU path for ComfyUI on Strix Halo
|
||
- [[Lemonade and sd.cpp]] — the ggml-family image-generation alternative
|
||
- [[Llama Cpp Runtimes]] — mainline vs charlie12345/ROCmFPX vs ik_llama.cpp: quantization + server fit
|
||
- [[Llama Cpp Code Diff]] — code-level quantization diff of the three llama.cpp runtimes
|
||
- [[SGLang]] — the structured-generation serving engine (llama.cpp/vLLM comparison + our fit)
|
||
- [[Backblaze Integration]] — off-site backup plan (future)
|
||
- [[Jellyfin]] — media server research brief
|
||
- [[ARR-Stack]] — automated download stack research brief
|
||
|
||
## LLM Fleet
|
||
|
||
The LLM inference fleet on the two hosts. Full index:
|
||
[serverAppConfigs-docs/LLM/index.md](LLM/index.md).
|
||
|
||
### Deployment and Topology
|
||
|
||
- [[deployment]] — unified deploy and update process for both servers
|
||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||
- [[freetoken-frank-default-small]] — the active frank-default-small runbook: FreeToken v0.1.2 serving Qwen3.6-35B-A3B-FP8 on Dankserver (2026-09-02)
|
||
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (restored 2026-09-01)
|
||
- [[agentionai-fast-imatrix]] — the ACTIVE frank-default-large runbook:
|
||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||
Q4_0_ROCMFP4_FAST) + own MTP draft head, build-qwen38 (2026-08-29)
|
||
- [[kingjones-strix-rocmfp4]] — retired frank-default-large runbook:
|
||
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B params, 113.47 GiB)
|
||
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
|
||
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
|
||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for frank-default-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
|
||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
|
||
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
|
||
- [[Dankserver System Specs]] — the RTX 3090 box hardware
|
||
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
|
||
- [[Dankclaw Improvements]] — host-level changes on the CIRU box
|
||
- [[frank-default-sightsound]] — ACTIVE FLM NPU sight+sound runbook on Dankclaw:
|
||
Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 7771; whole local
|
||
registry served on one port with per-request model routing; includes the
|
||
IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash
|
||
|
||
### Benchmarks and Sizing
|
||
|
||
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
|
||
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
|
||
- [[frank-default-small-benchmark]] — the small-model benchmark suite
|
||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
|
||
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
|
||
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
|
||
|
||
### Memory and Hardware
|
||
|
||
- [[RAM CPU Offloading in VLLM]] — KV cache swap between GPU VRAM and system RAM
|
||
- [[Hybrid CPU GPU Processing Dankserver]] — NVIDIA GPU compute with system RAM offload
|
||
- [[Shared Memory Architecture Dankclaw]] — AMD unified memory and ROCm configuration
|
||
- [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash
|
||
- [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU
|
||
|
||
### Retired Topology and History
|
||
|
||
- [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker
|
||
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP)
|
||
- [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint
|
||
- [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting
|
||
- [[ling-3.0-flash-trial]] — the Ling-3.0-Flash trial runbook
|
||
- [[ling-3.0-flash-vllm-plan]] — the Ling-3.0-Flash vLLM plan (retired)
|
||
|
||
### vLLM Reference (historical)
|
||
|
||
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
|
||
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
|
||
- [[vllm_research_findings]] — research findings from the vLLM era
|
||
|
||
### Pipeline Parallelism (archive)
|
||
|
||
- [[Ethernet Bandwidth Math]] — the bandwidth budget for the PP pair
|
||
- [[Expected Performance]] — expected PP throughput
|
||
- [[llama.cpp RPC]] — the RPC transport between head and worker
|
||
- [[Mixed-Vendor Experiments]] — cross-vendor PP experiments
|
||
- [[Qwen3.6 27B Spec Methods]] — spec-decoding methods for the 27B
|
||
- [[Sample Script Set]] — reference scripts for the PP pair
|
||
- [[vLLM Pipeline Parallelism]] — vLLM's PP implementation
|
||
|
||
## Update Log
|
||
|
||
- [[log]] — chronological record of knowledge-base changes
|
||
|
||
## See Also
|
||
|
||
- [AGENTS.md](../AGENTS.md) — repository rules
|
||
- [serverAppConfigs-docs/LLM/index.md](LLM/index.md) — the LLM fleet knowledge base
|
||
- [dankware Configuration Knowledgebase](~/.pi/agent/docs/index.md) — the "root" level system knowlegebase
|