Frank
219a097b39
Every payload swap a live endpoint has ever run is now one row in that endpoint's Eras table, with its rollback path in the same row. No per-attempt page, no per-swap ledger, nowhere else. - Added: LLM/frank-archon.md (radiance, 2x R9700 TP=2), LLM/frank-dragoon.md (halogen flash-server, 19 eras), LLM/frank-zealot.md (llama.cpp PrismML fork on the 3090). Each carries Current shape, Eras, Laws, wire contract, files. - Deleted: 45 per-payload runbooks -- the frank-dragoon line (11 including the pre-name halogen/kingjones/kairic/CIRU/eemin/agentionai/dflash2 pages and the benchmark page), the frank-zealot line (llama.cpp/vLLM/ik_llama/SGLang eras), the frank-archon line (vLLM/FreeToken/R9V/llama.cpp/radiance eras), plus frank-medium, frank-npu and the ubatch/PCIe one-offs that owned nothing. - Rewired: every wikilink and markdown-path reference to a folded page points at the endpoint page instead; the three Research see-also lists and the ubatch sweep's runbook link included. No anchored links existed in the set. - Regenerated both index pages as pure locators -- one row, one clause, no numbers -- so the docs_completeness locator law holds for all 153 pages. tests/docs_completeness.bats green: index links every page, no unresolved links. The pages moved to .AGENTTRASH/20261018-doc-dedupe/LLM/ (gitignored, and git history holds every byte).
20 KiB
20 KiB
serverAppConfigs — knowledge base
One fact, one owner. A page owns exactly one thing and everywhere else links
to it with a wikilink. Shape, payload, eras, laws and rollback for an inference
endpoint live on its endpoint page; scripts/<service>.env is the authority for
every number, and a page that disagrees with it is the bug. Payload swaps are
rows in an endpoint page's Eras table, never pages of their own. This index
is a locator: one row, one clause, no numbers.
Inference endpoints
| Page | Owns |
|---|---|
| Frank TTS | frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in… |
| frank-archon | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
| frank-dragoon | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
| frank-zealot | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
Retired endpoints
These services no longer exist, so no live page owns them; rollback provenance is inside each page. Every payload swap of a LIVE endpoint is an Eras row on that endpoint's page, not a page.
| Page | Owns |
|---|---|
| frank-medium-batch-ubatch | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
| frank-medium-llamacpp-dflash2 | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
| frank-medium-model-swap | How to swap the checkpoint behind the frank-medium endpoint (Dankserver :7770, vllm-radiance TP=2) — the streamlining… |
| frank-medium-radiance-mxfp4 | ACTIVE frank-medium runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
| frank-medium-rocmfpx | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
| frank-npu | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
| frank-redo-2026-08-17 | This page records the full redo of the inference fleet on 2026-08-17. |
| ikllama-ornith-dankserver | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
| kingjones-q8agent-rocmfpx | Status (historical, 2026-09-13): was to deploy on Dankclaw. |
| qwen3.8-spec-matrix | This page records the speculative-decoding benchmark history and the |
Dankclaw services (192.168.0.143)
| Page | Owns |
|---|---|
| ComfyUI | ComfyUI is a node-based AI image generation engine. |
| ComfyUI Qwen-Image Workflows | Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which… |
| Dankclaw Stack | Service inventory and routing overview for the Dankclaw AI, development, and observability host. |
| Frank TTS Engine | frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker,… |
| Gitea | Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service. |
| Gluetun | Gluetun is a lightweight VPN client container. |
| Hermes | Hermes is the self-improving AI agent from Nous Research. |
| Hermes-Dashboard | The Hermes web dashboard is a browser UI for managing the agent. |
| Hermes-WebUI | The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH… |
| Marreta | The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet… |
| Open Notebook | open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out —… |
| OpenWebUI | OpenWebUI is a self-hosted chat interface for LLMs. |
| Postgres | PostgreSQL is the database backend for Gitea. |
| SearXNG | SearXNG is a privacy-respecting metasearch engine. |
| SearXNG Providers | How Google and other upstream search engines get blocked behind the Gluetun |
| Seerr | Seerr is the unified request manager for the *arr stack (the maintained seerr-team/seerr successor to the archived… |
| Swap Configuration | The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit.… |
| Valkey | Valkey is the cache and limiter store for SearXNG. |
| Voice Packs | How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material… |
Dankserver services (192.168.0.235)
| Page | Owns |
|---|---|
| Calibre | Calibre is the e-book library manager that backs Calibre-Web. |
| Calibre-Web | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
| Copyparty | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
| Dankserver Stack | Dankserver (192.168.0.235) runs the media and library stack. |
| Entropy Volume | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
| Jellyfin | Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and… |
| mariadb | MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and… |
| Plex | Plex Media Server is a containerized media server on Dankserver: a podman |
| redis | Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup… |
| RomM | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
| RomM-Scraping | This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping… |
| Tautulli | Tautulli is the statistics, monitoring, and notification companion for |
FrankBook-Pro (192.168.0.112)
| Page | Owns |
|---|---|
| ARR-Stack | The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every… |
| lidarr | Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. |
| prowlarr | Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and… |
| qbittorrent | qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns,… |
| radarr | Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. |
| readarr | Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB… |
| sonarr | Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. |
Workstations
| Page | Owns |
|---|---|
| Noctalia and Niri | The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette… |
Cross-host infrastructure
| Page | Owns |
|---|---|
| Alertmanager | Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to… |
| Alloy | Alloy is Grafana's log and telemetry shipper. |
| Anubis | Anubis is the bot-challenge proxy that fronts the guarded public vhosts on |
| Authentik | Authentik is the single sign-on (SSO) and OpenID Connect (OIDC) provider for the |
| authentik-postgres | The PostgreSQL 16 database for Authentik — the homelab's SSO / |
| authentik-server | The Authentik server — the core, the admin API, and the embedded Caddy outpost. |
| authentik-worker | The Authentik worker — background tasks: event processing, provider |
| Caddy | Caddy is the single reverse proxy for both hosts. |
| FrankBook-Pro | FrankBook-Pro — the Mac mini running the *arr stack + qBittorrent inside a podman machine VM, gluetun-gated |
| Grafana | Grafana visualizes the metrics that Prometheus collects from every service. |
| Homepage | A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link… |
| Homepage Design Grilling | The decision log for the homepage design. |
| Homepage Status Probes | The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage… |
| LAN-DNS | The homelab gives every service a name under franksdank.cloud. |
| Loki | Loki is the log aggregation backend of the observability stack. |
| Node-Exporter | Node-Exporter exposes host metrics (CPU, RAM, disk, network) to Prometheus. |
| Observability | The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal… |
| Observability Provisioning | How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the… |
| Pi-hole | Pi-hole is the LAN DNS server for the homelab. |
| Prometheus | Prometheus collects metrics from every service on Dankclaw by scraping their /metrics endpoints, and feeds the… |
Repo architecture and policy
| Page | Owns |
|---|---|
| Container Network Namespaces | Which local services are addressable from which Podman network namespace on |
| Deploy State Triage | Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout,… |
| Deployment Flow | Service-scoped Git deployment for the two homelab hosts. |
| E2E Suite | tests/e2e/ drives the live fleet through a real browser: fresh contexts |
| Network Architecture | One Caddy fronts both hosts. |
| Quadlet Lifecycle | The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop… |
| Stale Signal Audit | The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers,… |
| Storage Policy | All persistent data uses bind mounts: a host directory mapped directly into the container. |
| Testing Policy | This repository tests its configuration at defined seams. |
Operations and incidents
| Page | Owns |
|---|---|
| Container Host Reachability | The two podman/netavark reachability quirks on the fleet — containers cannot dial unpublished host ports (LAN IP or… |
| Fleet Audit and Posture | Three standing audits — monitoring completeness (what should exist vs what is merely down), vhost proxy health… |
| Fleet Ops Playbook | Incident-shaped fleet procedures — LLM endpoint down with an abandoned GPU holder, scratch vLLM boot windows… |
| Payload and Disk Reclaim | Standing procedure for disk pressure on a fleet host — measure all buckets first, classify against the runbook index,… |
| Remote Job Runner | Detached long-job execution on fleet hosts with sentinel files (job.sh) — why ssh tail-watchers fail structurally,… |
| SSH and Remote Access | Passwordless SSH bootstrap without sshpass/expect, remote execution against fish login shells, and diagnosing a… |
Agent / harness
| Page | Owns |
|---|---|
| Agent Docs and Edits | Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline… |
| Command Center Dotfiles | The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite… |
| OMP Harness | The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via… |
Research and methodology
| Page | Owns |
|---|---|
| AI Job Search and Resume Tools | Self-hosted AI job-search and resume-formatting applications: the 2026 landscape, local-LLM fit, and a recommended… |
| Backblaze Integration | Off-site backup plan for the bind-mounted data on both hosts. |
| ComfyUI ROCm and Vulkan | Research into whether ComfyUI can use GPU acceleration — not just CPU — on the AMD Strix Halo APU (gfx1151) in… |
| Lemonade and sd.cpp | Why Lemonade Server can do image generation on Strix Halo when ComfyUI cannot. |
| Llama Cpp Code Diff | Code-level diff of mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp with a focus on quantization code, ggml… |
| Llama Cpp Runtimes | Compared mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp — positioning, quantization formats, code-level… |
| Local TTS and Voice Cloning | How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the… |
| Most Popular Self-Hosted AI Apps | The 15 most popular self-hosted AI applications of June–September 2026, ranked by GitHub stars, with homelab fit and… |
| PICO-8 Asset Standards | Production-grade standards for PICO-8 asset sheets — palette discipline, single perspective and light direction,… |
| Service Research Brief | Consolidated research from the frank-zealot research subagents (2026-08-14) plus the pi agent knowledge base. |
| SGLang | What SGLang is, how it works (RadixAttention, paged KV, overlap scheduler, structured output), how it compares to… |
LLM technique and hardware references
| Page | Owns |
|---|---|
| Dankclaw Improvements | This page collects concrete, sourced recommendations to improve Dankclaw |
| Dankclaw System Specs | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
| dankclaw-iommu-translation-off | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
| Dankserver System Specs | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
| dankserver-frontier-moe-feasibility | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
| dankserver-vram-growth | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
| dealign-ornith-crack | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
| deployment | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
| DFlash DSpark Speculative Decoding | DFlash and DSpark are the two newest speculative decoding methods. |
| Dual-R9700-Cooperation | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
| FastFlowLM NPU Inference | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
| FreeToken Builds | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
| freetoken-benchmark-sweep | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
| GGUF Compatibility | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
| Heterogeneous Split Triage | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
| Hybrid CPU GPU Processing Dankserver | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
| ling-3.0-flash-trial | bailingmoe3-support (commit db4480bc8, build b10411) at /home/dank/llama.cpp-ling. |
| ling-3.0-flash-vllm-plan | This page was the plan for a vLLM-ROCm deployment of jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native (INT4 W4A16, ~62… |
| Llama Cpp Multi-Device Splits | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
| Llama Cpp Serving Sizing | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
| LLM Bench Windows and Measurement | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
| LLM Performance Diagnosis | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
| Model Swap and Rollback | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
| PCIe Link and Atomics | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
| Quantization by Hardware Architecture | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
| Quantization Production | — |
| qwen3.8-strix-optimizations | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
| RAM CPU Offloading in VLLM | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
| RDNA4 Container Serving | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
| RDNA4 MoE Expert Offload VRAM Budget | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
| Running Qwen3.6 27B FP8 on Dankclaw | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
| Running Qwen3.6 35B A3B FP8 on Dankserver | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
| Shared Memory Architecture Dankclaw | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
| swift15-campaign-worklog | — |
| Tcclaviger Fork Ops | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
| ubatch-benchmark-large | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
| vLLM Boot Forensics and Config Interrogation | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
| VLLM Index | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
| VLLM Overview | vLLM is a fast and easy library for large language model inference and serving. |
| vLLM Porting and Compile Triage | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
| vllm_research_findings | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
| x4-pair-experiment | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
Pipeline Parallelism (series)
| Page | Owns |
|---|---|
| Ethernet Bandwidth Math | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
| Expected Performance | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
| llama.cpp RPC | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
| Mixed-Vendor Experiments | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
| Qwen3.6 27B Spec Methods | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
| Qwen3.8 PP Deployment | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
| Sample Script Set | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
| vLLM Pipeline Parallelism | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
Uncategorised
| Page | Owns |
|---|---|
| log | — |