diff --git a/serverAppConfigs-docs/Agent/Agent Docs and Edits.md b/serverAppConfigs-docs/Agent/Agent Docs and Edits.md index 8e58062..0104539 100644 --- a/serverAppConfigs-docs/Agent/Agent Docs and Edits.md +++ b/serverAppConfigs-docs/Agent/Agent Docs and Edits.md @@ -1,7 +1,7 @@ --- type: reference title: Agent Docs and Edits -description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each. +description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each." tags: [agents, docs, markdown, hashline, edit-recovery] timestamp: 2026-10-18T00:00:00Z --- diff --git a/serverAppConfigs-docs/Agent/Command Center Dotfiles.md b/serverAppConfigs-docs/Agent/Command Center Dotfiles.md index de95761..617c658 100644 --- a/serverAppConfigs-docs/Agent/Command Center Dotfiles.md +++ b/serverAppConfigs-docs/Agent/Command Center Dotfiles.md @@ -1,7 +1,7 @@ --- type: reference title: Command Center Dotfiles -description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas. +description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas." tags: [dotfiles, omp, deploy, rsync, blankpad] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Agent/OMP Harness.md b/serverAppConfigs-docs/Agent/OMP Harness.md index 48eafda..4e44a74 100644 --- a/serverAppConfigs-docs/Agent/OMP Harness.md +++ b/serverAppConfigs-docs/Agent/OMP Harness.md @@ -1,7 +1,7 @@ --- type: reference title: OMP Harness -description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean. +description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits." tags: [omp, harness, compaction, models-yml, context, samplers] timestamp: 2026-10-18T00:00:00Z --- diff --git a/serverAppConfigs-docs/Agent/index.md b/serverAppConfigs-docs/Agent/index.md index d042b3c..26c4697 100644 --- a/serverAppConfigs-docs/Agent/index.md +++ b/serverAppConfigs-docs/Agent/index.md @@ -1,23 +1,12 @@ ---- -type: index -description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home. -tags: [agent, index] -timestamp: 2026-10-05T00:00:00-07:00 ---- - # Agent Pages about the agent's own tooling and context — how omp is configured, how docs get edited safely, and how the agent's home is deployed. -- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via - `compat.extraBody`, preamble and skill-library audits -- [[Agent Docs and Edits]] — doc-structure audits after parallel edits; - hashline edit recovery -- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync - transport, override packages +- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits. +- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each. +- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas. ## Related -- [[Testing Policy]] — the seams these pages' checks ride - [LLM](../LLM/index.md) — the fleet these agents operate diff --git a/serverAppConfigs-docs/Architecture/Container Network Namespaces.md b/serverAppConfigs-docs/Architecture/Container Network Namespaces.md index 69ef69d..e07af61 100644 --- a/serverAppConfigs-docs/Architecture/Container Network Namespaces.md +++ b/serverAppConfigs-docs/Architecture/Container Network Namespaces.md @@ -1,8 +1,7 @@ --- type: reference title: Container Network Namespaces -description: Which local services are addressable from which Podman network namespace on - Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners. +description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits." tags: - networking - podman diff --git a/serverAppConfigs-docs/Architecture/Deploy State Triage.md b/serverAppConfigs-docs/Architecture/Deploy State Triage.md index 4bdc511..6ab442a 100644 --- a/serverAppConfigs-docs/Architecture/Deploy State Triage.md +++ b/serverAppConfigs-docs/Architecture/Deploy State Triage.md @@ -1,7 +1,7 @@ --- type: reference title: Deploy State Triage -description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints. +description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy." tags: [deployment, deploy-state, systemd, podman, triage] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Architecture/Deployment Flow.md b/serverAppConfigs-docs/Architecture/Deployment Flow.md index 2b56363..44195be 100644 --- a/serverAppConfigs-docs/Architecture/Deployment Flow.md +++ b/serverAppConfigs-docs/Architecture/Deployment Flow.md @@ -1,5 +1,5 @@ --- -type: guide +type: howto title: Deployment Flow description: Service-scoped Git deployment for the two homelab hosts. tags: diff --git a/serverAppConfigs-docs/Architecture/E2E Suite.md b/serverAppConfigs-docs/Architecture/E2E Suite.md index 179bd23..a03d3f0 100644 --- a/serverAppConfigs-docs/Architecture/E2E Suite.md +++ b/serverAppConfigs-docs/Architecture/E2E Suite.md @@ -1,6 +1,6 @@ --- type: reference -description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts" +description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional." tags: - architecture - e2e diff --git a/serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md b/serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md index c5f1b24..eaa5769 100644 --- a/serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md +++ b/serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md @@ -1,7 +1,7 @@ --- type: reference title: Quadlet Lifecycle -description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth. +description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval." tags: [podman, quadlet, systemd, boot-persistence, compose, drift] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Architecture/Stale Signal Audit.md b/serverAppConfigs-docs/Architecture/Stale Signal Audit.md index 37c16cb..6ef56e3 100644 --- a/serverAppConfigs-docs/Architecture/Stale Signal Audit.md +++ b/serverAppConfigs-docs/Architecture/Stale Signal Audit.md @@ -1,7 +1,7 @@ --- type: reference title: Stale Signal Audit -description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time. +description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run." tags: [testing, observability, audit, signals, gates] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md b/serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md index 70ee470..97a8f8b 100644 --- a/serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md +++ b/serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md @@ -1,6 +1,6 @@ --- type: howto -description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next." +description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next." tags: - dankclaw - comfyui diff --git a/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md b/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md index 55ae8f3..981ea19 100644 --- a/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md +++ b/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md @@ -1,6 +1,6 @@ --- type: reference -description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose." +description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes." tags: - dankclaw - tts diff --git a/serverAppConfigs-docs/Dankclaw/Frank TTS.md b/serverAppConfigs-docs/Dankclaw/Frank TTS.md index a3c3df0..b0344c0 100644 --- a/serverAppConfigs-docs/Dankclaw/Frank TTS.md +++ b/serverAppConfigs-docs/Dankclaw/Frank TTS.md @@ -1,11 +1,13 @@ --- type: reference -description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role." +description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role." tags: + - endpoint - dankclaw - tts - voice - quadlet +timestamp: 2026-10-19 --- # Frank TTS @@ -202,3 +204,4 @@ harness bundle and measured the day of the flip: | First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) | | The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` | | A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] | +--- diff --git a/serverAppConfigs-docs/Dankclaw/Gitea.md b/serverAppConfigs-docs/Dankclaw/Gitea.md index f8abd5d..dac5aad 100644 --- a/serverAppConfigs-docs/Dankclaw/Gitea.md +++ b/serverAppConfigs-docs/Dankclaw/Gitea.md @@ -1,5 +1,5 @@ --- -type: configuration +type: reference title: Gitea description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service. tags: diff --git a/serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md b/serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md index d1b8159..f790365 100644 --- a/serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md +++ b/serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: Hermes-WebUI (Hermex server) -description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost). +description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN." tags: - hermes - hermes-webui diff --git a/serverAppConfigs-docs/Dankclaw/Marreta.md b/serverAppConfigs-docs/Dankclaw/Marreta.md index 71ce484..308a056 100644 --- a/serverAppConfigs-docs/Dankclaw/Marreta.md +++ b/serverAppConfigs-docs/Dankclaw/Marreta.md @@ -1,7 +1,7 @@ --- type: reference title: Marreta on Dankclaw -description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud. +description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain." tags: - dankclaw - marreta diff --git a/serverAppConfigs-docs/Dankclaw/Open Notebook.md b/serverAppConfigs-docs/Dankclaw/Open Notebook.md index 96cb5ca..04f4cd1 100644 --- a/serverAppConfigs-docs/Dankclaw/Open Notebook.md +++ b/serverAppConfigs-docs/Dankclaw/Open Notebook.md @@ -1,6 +1,6 @@ --- type: reference -description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path." +description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house." tags: - dankclaw - open-notebook diff --git a/serverAppConfigs-docs/Dankclaw/Seerr.md b/serverAppConfigs-docs/Dankclaw/Seerr.md index 291890a..f347c25 100644 --- a/serverAppConfigs-docs/Dankclaw/Seerr.md +++ b/serverAppConfigs-docs/Dankclaw/Seerr.md @@ -1,6 +1,6 @@ --- type: reference -description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)." +description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line." tags: - dankclaw - seerr @@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to authentik's own user UI instead of Seerr. Both backfilled live: the three standard flows copied from a known-good provider, plus an application (`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep -(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two -provisioning rules this incident established. +(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the +two provisioning rules this incident established. ## *arr wiring diff --git a/serverAppConfigs-docs/Dankclaw/Swap Configuration.md b/serverAppConfigs-docs/Dankclaw/Swap Configuration.md index e057e24..eab3eae 100644 --- a/serverAppConfigs-docs/Dankclaw/Swap Configuration.md +++ b/serverAppConfigs-docs/Dankclaw/Swap Configuration.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: Dankclaw Swap Configuration -description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat. +description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat." tags: - dankclaw - host diff --git a/serverAppConfigs-docs/Dankclaw/Voice Packs.md b/serverAppConfigs-docs/Dankclaw/Voice Packs.md index f25efc8..aebeafa 100644 --- a/serverAppConfigs-docs/Dankclaw/Voice Packs.md +++ b/serverAppConfigs-docs/Dankclaw/Voice Packs.md @@ -1,6 +1,6 @@ --- type: reference -description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes." +description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift." tags: - dankclaw - frank-tts diff --git a/serverAppConfigs-docs/Dankserver/Copyparty.md b/serverAppConfigs-docs/Dankserver/Copyparty.md index 951102a..87fa51c 100644 --- a/serverAppConfigs-docs/Dankserver/Copyparty.md +++ b/serverAppConfigs-docs/Dankserver/Copyparty.md @@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e ## Security Advisories -| Version | CVE | Issue | -|---------|-----|-------| -| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume | -| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion | -| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload | +| Version | Advisory | CVE | Issue | +|---------|----------|-----|-------| +| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) | +| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion | +| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) | -Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. +Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`. ## Gotchas @@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters - [[Dankserver Stack]] - [[Caddy]] — TLS termination and authentication in front - [[ARR-Stack]] — Copyparty as a viewer alongside automated download management + +## Citations + +- GitHub Security Advisories, `9001/copyparty` — + https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19); + each row's patched version read from `first_patched_version` in the + `https://api.github.com/advisories/` response. diff --git a/serverAppConfigs-docs/Dankserver/Dankserver Stack.md b/serverAppConfigs-docs/Dankserver/Dankserver Stack.md index e8d51fe..31f3972 100644 --- a/serverAppConfigs-docs/Dankserver/Dankserver Stack.md +++ b/serverAppConfigs-docs/Dankserver/Dankserver Stack.md @@ -1,6 +1,6 @@ --- type: reference -description: "Dankserver (192.168.0.235) runs the media and library stack." +description: "Service inventory and routing overview for the media and library host." tags: - dankserver timestamp: 2026-09-03 diff --git a/serverAppConfigs-docs/Dankserver/Jellyfin.md b/serverAppConfigs-docs/Dankserver/Jellyfin.md index 8dd05fd..1af373a 100644 --- a/serverAppConfigs-docs/Dankserver/Jellyfin.md +++ b/serverAppConfigs-docs/Dankserver/Jellyfin.md @@ -1,7 +1,7 @@ --- type: reference title: Jellyfin -description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status. +description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status." tags: [jellyfin, dankserver, media-server] timestamp: 2026-09-03T00:00:00Z --- diff --git a/serverAppConfigs-docs/Dankserver/RomM-Scraping.md b/serverAppConfigs-docs/Dankserver/RomM-Scraping.md index 2e98e8a..8a573df 100644 --- a/serverAppConfigs-docs/Dankserver/RomM-Scraping.md +++ b/serverAppConfigs-docs/Dankserver/RomM-Scraping.md @@ -1,6 +1,6 @@ --- type: reference -description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver." +description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library." tags: - dankserver - romm diff --git a/serverAppConfigs-docs/Dankserver/mariadb.md b/serverAppConfigs-docs/Dankserver/mariadb.md index 0776c72..256908a 100644 --- a/serverAppConfigs-docs/Dankserver/mariadb.md +++ b/serverAppConfigs-docs/Dankserver/mariadb.md @@ -1,7 +1,7 @@ --- type: reference title: MariaDB (RomM Backend) -description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path. +description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path." tags: [mariadb, romm, dankserver] timestamp: 2026-09-03T00:00:00Z --- diff --git a/serverAppConfigs-docs/Dankserver/redis.md b/serverAppConfigs-docs/Dankserver/redis.md index fcdc1ea..ecb36c3 100644 --- a/serverAppConfigs-docs/Dankserver/redis.md +++ b/serverAppConfigs-docs/Dankserver/redis.md @@ -1,7 +1,7 @@ --- type: reference title: Redis (RomM Cache / Queue) -description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path. +description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path." tags: [redis, romm, dankserver] timestamp: 2026-09-03T00:00:00Z --- diff --git a/serverAppConfigs-docs/Desktop/Noctalia and Niri.md b/serverAppConfigs-docs/Desktop/Noctalia and Niri.md index 3fafdd4..097c5c3 100644 --- a/serverAppConfigs-docs/Desktop/Noctalia and Niri.md +++ b/serverAppConfigs-docs/Desktop/Noctalia and Niri.md @@ -1,7 +1,7 @@ --- type: reference title: Noctalia and Niri -description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning. +description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning." tags: [desktop, noctalia, niri, blankpad, workstation, ghostty] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md b/serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md index d4147bc..4ffaf42 100644 --- a/serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md +++ b/serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md @@ -1,7 +1,7 @@ --- type: reference title: "*arr Stack" -description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas." +description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas." tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver] timestamp: 2026-09-04T00:00:00Z --- diff --git a/serverAppConfigs-docs/FrankBook-Pro/index.md b/serverAppConfigs-docs/FrankBook-Pro/index.md index bcf0d6a..ee926d3 100644 --- a/serverAppConfigs-docs/FrankBook-Pro/index.md +++ b/serverAppConfigs-docs/FrankBook-Pro/index.md @@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]] ## *arr Stack -- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client -- [[sonarr]] — TV show management -- [[radarr]] — movie management -- [[lidarr]] — music management -- [[readarr]] — ebook/audiobook management -- [[prowlarr]] — indexer management -- [[qbittorrent]] — torrent download client +- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas. +- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. +- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. +- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. +- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client. +- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas. +- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe. diff --git a/serverAppConfigs-docs/FrankBook-Pro/prowlarr.md b/serverAppConfigs-docs/FrankBook-Pro/prowlarr.md index a79e79c..b204f8a 100644 --- a/serverAppConfigs-docs/FrankBook-Pro/prowlarr.md +++ b/serverAppConfigs-docs/FrankBook-Pro/prowlarr.md @@ -1,7 +1,7 @@ --- type: reference title: Prowlarr -description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas. +description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas." tags: [prowlarr, arr-stack, frankbook-pro] timestamp: 2026-09-03T00:00:00Z --- diff --git a/serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md b/serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md index 7cbf6f2..65a8446 100644 --- a/serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md +++ b/serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md @@ -1,7 +1,7 @@ --- type: reference title: qBittorrent -description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe. +description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe." tags: [qbittorrent, arr-stack, frankbook-pro, gluetun] timestamp: 2026-09-03T00:00:00Z --- diff --git a/serverAppConfigs-docs/FrankBook-Pro/readarr.md b/serverAppConfigs-docs/FrankBook-Pro/readarr.md index 56f5693..1737792 100644 --- a/serverAppConfigs-docs/FrankBook-Pro/readarr.md +++ b/serverAppConfigs-docs/FrankBook-Pro/readarr.md @@ -1,18 +1,19 @@ --- type: reference title: Readarr -description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client. +description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client." tags: [readarr, arr-stack, frankbook-pro] timestamp: 2026-09-03T00:00:00Z --- # Readarr -> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its -> metadata became unusable and the community migration to Open Library stalled. -> There is no upstream development. Alternatives: the Bookshelf community fork, -> or the Calibre content server. This instance still runs on a pinned image -> tag; see below. +> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired +> Readarr: its metadata became unusable and the community migration to Open +> Library stalled. There is no upstream development — the GitHub repository is +> archived and its last push is 2025-06-27. Alternatives: the Bookshelf +> community fork, or the Calibre content server. This instance still runs on a +> pinned image tag; see below. Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on **FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine — @@ -69,9 +70,9 @@ Restart=always ## Gotchas -- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before - the final tags). The pinned `amd64-1.0.980` tag exists precisely because - `:latest` does not exist for this image on LSIO. +- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x + before the final tags). The pinned `amd64-1.0.980` tag exists precisely + because `:latest` does not exist for this image on LSIO. - Calibre integration is folder-based, not API-based. Readarr drops files into the books root which [[Calibre-Web]] serves. - Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph @@ -81,3 +82,10 @@ Restart=always - [[ARR-Stack]] — full *arr stack documentation - [[Calibre-Web]] — ebook web frontend that serves Readarr downloads + +## Citations + +- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr + 2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19) +- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 — + https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19) diff --git a/serverAppConfigs-docs/Infrastructure/Alertmanager.md b/serverAppConfigs-docs/Infrastructure/Alertmanager.md index f33aefd..2984935 100644 --- a/serverAppConfigs-docs/Infrastructure/Alertmanager.md +++ b/serverAppConfigs-docs/Infrastructure/Alertmanager.md @@ -1,6 +1,6 @@ --- type: reference -description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..." +description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay." tags: - infrastructure timestamp: 2026-09-17 diff --git a/serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md b/serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md new file mode 100644 index 0000000..6139884 --- /dev/null +++ b/serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md @@ -0,0 +1,230 @@ +--- +type: reference +description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO." +tags: + - infrastructure + - authentik +timestamp: 2026-10-19 +--- + +# Authentik Outpost Reference + +Configuration lookup for the [[Authentik]] deployment: the live proxy-provider +inventory behind the guarded vhosts, the provisioning rules the embedded outpost +enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the +self-serve user-settings flow, and the live database shape. What the service is, +how it is deployed, wired and operated is [[Authentik]]; the three core +containers are [[Authentik Processes]]. + +## Live proxy-provider inventory + +Live provider inventory (read from the `authentik-postgres` container with +read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost +`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9 +members in `authentik_outposts_outpost_providers`), all with +`internal_host = http://caddy:8017`, all carrying the **same three flow +UUIDs** (`default-authentication-flow`, +`default-provider-authorization-implicit-consent`, +`default-provider-invalidation-flow`), and each backed by an Application +(the `slug` column below): + +| Provider pk | name | external_host | app slug | +|---|---|---|---| +| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy | +| 2 | search-proxy | `https://search.franksdank.cloud` | search | +| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus | +| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm | +| 6 | files-proxy | `https://files.franksdank.cloud` | files | +| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki | +| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard | +| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard | +| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta | + +That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so +the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table: +**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and +**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3` +returns zero rows, so it is deleted, not "dormant"; its vhost survived but is +exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties, +not proxy providers, so the proxy outpost does not deserialize them: pk 9 +`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC` +(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule +below rejects), and pk 14 `Open WebUI` +(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on +`external_host` (the runbook-recipe style) and bare origins both intercept +correctly — the outpost matches on hostname. + +## Provisioning rules for proxy providers + +Two provisioning rules, each verified live by an outage: + +1. **All three flows are required.** A ProxyProvider created via raw API/ORM + with any of `authentication_flow`/`authorization_flow`/`invalidation_flow` + null wedges the embedded outpost's config fetch: the Rust side rejects the + whole config JSON (serde: `invalid type: null, expected a formatted UUID + string`), logs `Failed to fetch outpost from API, retrying in 3 seconds` + forever, and **everything behind the outpost dies at the next outpost + reload** — including hosts whose providers were fine (verified 2026-09-02 + when three providers provisioned without any flows took down every guarded + vhost after a server restart). Backfill all three flows (copy from pk 1: + `default-authentication-flow`, + `default-provider-authorization-implicit-consent` (the implicit-consent + authorization flow, `default-invalidation-flow`) and the retry loop + recovers on its next 3-second cycle. The silent killer: while the outpost + process keeps its last good in-memory config, everything works — the + outage only surfaces on restart/reload. +2. **Every proxy provider needs an attached Application.** The outpost + resolves the post-login redirect through the provider's application; a + provider with no app completes the login (authentik session established) + but bounces to authentik's own user UI (`/if/user/#/library`) instead of + the app. No error is logged anywhere — the SSO "works" and the app never + appears. Create the Application with `provider_id` set (the UI does this + automatically; raw API/ORM drives must do it explicitly). + +## OAuth2 relying-party quirks (this fork, verified live 2026-08-31) + +The Grafana RP provider (`grafana`, pk 9, confidential client) was created and +verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider +implementation diverges from upstream in ways that bit us; all verified live: + +- **Application row required.** `OAuth2Provider.objects.create()` does not + create the single-table-inheritance `Application` row; without it the flow + executor can never complete an authorize request. Create both rows. +- **`grant_types` defaults to empty.** Until the provider's `grant_types` + contains `"authorization_code"` (enum value with underscore), the token + endpoint rejects every exchange with `invalid_request`. +- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes + `parse_token_request` see no `grant_type` → `unsupported_grant_type`; + POST-body client secrets are not accepted — authenticate the confidential + client with `Authorization: Basic base64(client_id:client_secret)`. +- **Provider resolution is by `client_id` query param** on the global + `/application/o/authorize/` route; per-slug authorize paths do not exist in + this build. Token/userinfo live at `/application/o/token/` and + `/application/o/userinfo/?api=1`. +- **Property-mapping evaluator is broken → userinfo returns `sub` only.** + `PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to + `BaseEvaluator.evaluate(expression_source)`, so the first context kwarg + becomes the *expression source* instead of the context; every call raises a + swallowed `PropertyMappingExpressionException` and the claim is skipped. + Scope-grant filtering has the same class of bug: `ScopeMapping` has no + provider FK (linkage lives in the `ProviderPropertyMapping` through model), + so `filter(provider=...)` always yields an empty queryset and tokens asked + to carry scopes get zero. Net effect: userinfo/introspection answers with + `{sub}` only — email/groups/entitlements claims cannot be delivered by this + build until the evaluator is patched or upgraded. Clients consuming this + provider must key identity off `sub`. +- Scope mappings for this provider exist — `grafana-email-scope` and + `grafana-profile-scope` are both rows in `authentik_core_propertymapping` + (read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment + the evaluator bug is fixed; no config change is needed then. + +## LDAP provider objects (provisioned via `ak shell` ORM) + +The LDAP outpost's own deployment — unit, image digest, published ports, token +env var — is in [[Authentik]]. The objects it serves: + +Read from the live database (read-only `SELECT`s against the +`authentik-postgres` container, 2026-10-18): + +| Object | Value | +|---|---| +| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` | +| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* | +| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* | +| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` | +| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) | +| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) | +| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) | + +Hard-won rules (the first is now contested by the live database): + +- **Token intent: contested, do not trust either value blindly.** An earlier + revision asserted that a bind token with `intent=app_password` is rejected by + the flow's password stage with `Invalid password` and only + `intent=verification` passes. The live token for `svc-jellyfin-ldap` is + `app_password` (`authentik_core_token`, 2026-10-18), so either the rule is + wrong or the current Jellyfin bind path is not that token. Nothing observable + settles it today: the LDAP outpost's retained log holds only its 5-minute + `initialized direct binder` / `Update providers` config cycles and no bind + activity *(unverified 2026-10-18: no bind was performed — reproducing it + requires the bind secret)*. Re-verify with a real bind before changing the + token's intent either way. +- **Search needs `search_full_directory`.** Without the RBAC grant the + outpost returns only the bind user itself — a `(memberOf=…)` filter then + matches zero entries and the plugin logs `Found no users matching X`. + The permission is object-level on the provider; grant it via an RBAC role + and retest. +- `ak shell` import path for the provider model in 2026.8: + `from authentik.providers.ldap.models import LDAPProvider`. + *(unverified 2026-10-18: importing it outside a configured Django shell fails + with `ImproperlyConfigured`, and no `ak shell` was driven against the live + auth service for this check)* + +## Self-serve user-settings flow (password change) + +Users change their own account details through the `default-user-settings-flow` +(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the +target of the homepage footer's "account" link. It is a `stage_configuration` +designated flow, so it renders every bound `ConfigurableStage` for the logged-in +user. + +Stages, in execution order (the `FlowStageBinding.order` field, not the M2M +insertion order): + +| order | stage | type | what it does | +|---|---|---|---| +| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** | +| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user | + +**Password change is a prompt field, not a stage.** The `UserWriteStage` +writes whatever the preceding `PromptStage` collected: for the key `password` +it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix +is a `password` field on the `default-user-settings` prompt +(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a +`PasswordStage`. A `PasswordStage` is an *authentication* stage: its +`validate_password` runs `authenticate()` against the backends to check the +*current* password; it never sets a new one. (A `PasswordStage` was added on +2026-09-25 and removed the same day for exactly this reason.) + +Two ORM gotchas: `Flow.stages.add()` fails on this through model +(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is +MTI), so create the `FlowStageBinding` row directly with an explicit `order`; +and a prompt field is a `Prompt` row linked through `PromptStageField` +(`prompt_stage` + `prompt`; the order comes from `Prompt.order`). + +**A blank password must not set an empty password.** `set_password("")` would +make the account log in with no password, so a validation policy +(`default-user-settings-no-empty-password`, bound to the prompt via +`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]` +is `""` and allows it when set. Verified with the `PolicyEvaluator` +(empty → deny, set → allow, absent → allow) and end-to-end by driving +`UserWriteStageView.update_user` on a throwaway user (new password valid, old +invalid). + +## Live database shape + +For live DB truth prefer the postgres container, and it works without any +credential — the local socket is trusted, so a read-only audit is just +`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -` +with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify +before guessing: 2026.x renamed both tables and columns, and the models are +MTI all the way down — `authentik_core_provider` holds `id`, `name` and the +three `*_flow_id` columns; the proxy table is +`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column +really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2 +table in 2026.8), the LDAP table is +`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`; +`authentik_core_application` attaches to a provider through `provider_id` +(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost` +is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError` +is the same story from the ORM side: `type` lives in the subclass tables. +Never select secret columns (`client_secret`, `key`, `password`) into a log. + +## Cross-references + +- [[Authentik]] — what the service is, its deployment, wiring, operations and + failure modes +- [[Authentik Processes]] — the postgres, server and worker containers +- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners +- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin +- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider diff --git a/serverAppConfigs-docs/Infrastructure/Authentik Processes.md b/serverAppConfigs-docs/Infrastructure/Authentik Processes.md new file mode 100644 index 0000000..a911a02 --- /dev/null +++ b/serverAppConfigs-docs/Infrastructure/Authentik Processes.md @@ -0,0 +1,76 @@ +--- +type: reference +description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes." +tags: + - infrastructure + - authentik + - postgres +timestamp: 2026-10-19 +--- + +# Authentik Processes + +The three core containers of the [[Authentik]] stack on Dankclaw. The two +outpost units are not covered here — they live on [[Authentik]] and +[[Authentik Outpost Reference]]. Server and worker run the same image; the +`command` selects the role (`server` / `worker`). Data persists under +`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named +volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet +standardization; the retired stubs this page replaces recorded the pre-move +paths `/home/dank/authentik/data` (server and worker) and +`/home/dank/authentik/postgres` (database). + +## authentik-postgres + +The PostgreSQL 16 database for Authentik — the only stateful store in the stack +(users, flows, sessions, events). Sessions, caching, and WebSockets moved to +PostgreSQL in 2025.10 (Redis removed). + +- **Image:** `docker.io/library/postgres:16-alpine` +- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named + volumes) +- **Health:** `pg_isready -d authentik -U authentik` +- **Retired wording:** the stub called this a "Compose service … in + `dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet. +- The side-by-side with `gitea-postgres` (network, published ports, secret + source, boot persistence) is owned by [[Postgres]]. + +## authentik-server + +The Authentik server — the core, the admin API, and the embedded Caddy outpost. +It also serves the login page at `https://auth.franksdank.cloud`. + +- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` + (`authentik-server.service`) +- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`) +- **Port:** `9000` (published) — the outpost, the admin API and the login UI all + live here +- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the + worker) +- **Health:** recorded by the retired stub as + `http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations + owns the live probe. +- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts" + proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy + integration. + +## authentik-worker + +The Authentik worker — background tasks: event processing, provider +synchronization, scheduled flows, and certificate renewal. + +- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` + (`authentik-worker.service`) +- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`) +- **Runs as:** `0:0` (rootless: container root maps to the host user that owns + the data dir) +- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the + server) +- **No port** — the worker has no HTTP endpoint. + +## Cross-references + +- [[Authentik]] — what the service is, its deployment, wiring, operations and + failure modes +- [[Authentik Outpost Reference]] — provider, flow and token configuration +- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side diff --git a/serverAppConfigs-docs/Infrastructure/Authentik.md b/serverAppConfigs-docs/Infrastructure/Authentik.md index 78d10e5..2d1578f 100644 --- a/serverAppConfigs-docs/Infrastructure/Authentik.md +++ b/serverAppConfigs-docs/Infrastructure/Authentik.md @@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) — they handle their own session. Grafana is the exception that is both: it is a guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2 relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`, -`dankclaw/Caddyfile:374-377`; live provider table below). +`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]). - **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`; the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`, @@ -35,6 +35,13 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`, pre-quadlet era, retired. - **LDAP outpost:** serves LDAP for Jellyfin — see the section below. +**Split off from this page:** [[Authentik Processes]] owns the per-container +facts for `authentik-postgres`, `authentik-server` and `authentik-worker`; +[[Authentik Outpost Reference]] owns the provider/flow/token lookup material — +the live proxy-provider inventory, the proxy-provider provisioning rules, the +OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow +design, and the live database shape. + ## Why Authentik The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was @@ -100,23 +107,10 @@ contract: The chain terminates there — nothing dials back into Authentik from `:8017`. -## The three compose services +## The stack's processes -| Service | Role | Image | -|---|---|---| -| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` | -| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` | -| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` | - -- **Postgres** is the only stateful store. Sessions, caching, and WebSockets - moved to PostgreSQL in 2025.10 (Redis removed). -- **Server** and **worker** run the same image; the `command` selects the role - (`server` / `worker`). -- The worker runs as `0:0` (rootless: container root maps to the host user that - owns the data dir). -- Data persists under `/home/dank/Containers/authentik/{data,postgres}` - (bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/` - in the 2026-09-01 quadlet standardization). +Per-container facts — unit path, image and role, port, run-as, data directory, +health probe — are owned by [[Authentik Processes]]. ## Caddy integration @@ -142,104 +136,9 @@ routing table, and the Authentik proxy-provider list must be the same set of hosts. A guarded host with no provider 404s at login time; a provider with no DNS record breaks off-LAN clients. -Live provider inventory (read from the `authentik-postgres` container with -read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost -`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9 -members in `authentik_outposts_outpost_providers`), all with -`internal_host = http://caddy:8017`, all carrying the **same three flow -UUIDs** (`default-authentication-flow`, -`default-provider-authorization-implicit-consent`, -`default-provider-invalidation-flow`), and each backed by an Application -(the `slug` column below): - -| Provider pk | name | external_host | app slug | -|---|---|---|---| -| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy | -| 2 | search-proxy | `https://search.franksdank.cloud` | search | -| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus | -| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm | -| 6 | files-proxy | `https://files.franksdank.cloud` | files | -| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki | -| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard | -| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard | -| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta | - -That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so -the lockstep rule holds today. Two corrections against the 2026-09-02 table: -**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and -**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3` -returns zero rows, so it is deleted, not "dormant"; its vhost survived but is -exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties, -not proxy providers, so the proxy outpost does not deserialize them: pk 9 -`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC` -(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule -below rejects), and pk 14 `Open WebUI` -(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on -`external_host` (the runbook-recipe style) and bare origins both intercept -correctly — the outpost matches on hostname. - -Two provisioning rules, each verified live by an outage: - -1. **All three flows are required.** A ProxyProvider created via raw API/ORM - with any of `authentication_flow`/`authorization_flow`/`invalidation_flow` - null wedges the embedded outpost's config fetch: the Rust side rejects the - whole config JSON (serde: `invalid type: null, expected a formatted UUID - string`), logs `Failed to fetch outpost from API, retrying in 3 seconds` - forever, and **everything behind the outpost dies at the next outpost - reload** — including hosts whose providers were fine (verified 2026-09-02 - when three providers provisioned without any flows took down every guarded - vhost after a server restart). Backfill all three flows (copy from pk 1: - `default-authentication-flow`, - `default-provider-authorization-implicit-consent` (the implicit-consent - authorization flow, `default-invalidation-flow`) and the retry loop - recovers on its next 3-second cycle. The silent killer: while the outpost - process keeps its last good in-memory config, everything works — the - outage only surfaces on restart/reload. -2. **Every proxy provider needs an attached Application.** The outpost - resolves the post-login redirect through the provider's application; a - provider with no app completes the login (authentik session established) - but bounces to authentik's own user UI (`/if/user/#/library`) instead of - the app. No error is logged anywhere — the SSO "works" and the app never - appears. Create the Application with `provider_id` set (the UI does this - automatically; raw API/ORM drives must do it explicitly). - - -## OAuth2 relying-party quirks (this fork, verified live 2026-08-31) - -The Grafana RP provider (`grafana`, pk 9, confidential client) was created and -verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider -implementation diverges from upstream in ways that bit us; all verified live: - -- **Application row required.** `OAuth2Provider.objects.create()` does not - create the single-table-inheritance `Application` row; without it the flow - executor can never complete an authorize request. Create both rows. -- **`grant_types` defaults to empty.** Until the provider's `grant_types` - contains `"authorization_code"` (enum value with underscore), the token - endpoint rejects every exchange with `invalid_request`. -- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes - `parse_token_request` see no `grant_type` → `unsupported_grant_type`; - POST-body client secrets are not accepted — authenticate the confidential - client with `Authorization: Basic base64(client_id:client_secret)`. -- **Provider resolution is by `client_id` query param** on the global - `/application/o/authorize/` route; per-slug authorize paths do not exist in - this build. Token/userinfo live at `/application/o/token/` and - `/application/o/userinfo/?api=1`. -- **Property-mapping evaluator is broken → userinfo returns `sub` only.** - `PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to - `BaseEvaluator.evaluate(expression_source)`, so the first context kwarg - becomes the *expression source* instead of the context; every call raises a - swallowed `PropertyMappingExpressionException` and the claim is skipped. - Scope-grant filtering has the same class of bug: `ScopeMapping` has no - provider FK (linkage lives in the `ProviderPropertyMapping` through model), - so `filter(provider=...)` always yields an empty queryset and tokens asked - to carry scopes get zero. Net effect: userinfo/introspection answers with - `{sub}` only — email/groups/entitlements claims cannot be delivered by this - build until the evaluator is patched or upgraded. Clients consuming this - provider must key identity off `sub`. -- Scope mappings for this provider exist — `grafana-email-scope` and - `grafana-profile-scope` are both rows in `authentik_core_propertymapping` - (read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment - the evaluator bug is fixed; no config change is needed then. +The live proxy-provider inventory (the per-vhost `external_host` → app mapping), +the two provisioning rules the outpost enforces, and this fork's OAuth2 +relying-party quirks are reference lookup: [[Authentik Outpost Reference]]. ## LDAP outpost (Jellyfin auth, live 2026-09-01) @@ -267,44 +166,9 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect (2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN` from the host `dankclaw/.env` (git-ignored). -### Authentik-side objects (provisioned via `ak shell` ORM) - -Read from the live database (read-only `SELECT`s against the -`authentik-postgres` container, 2026-10-18): - -| Object | Value | -|---|---| -| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` | -| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* | -| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* | -| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` | -| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) | -| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) | -| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) | - -Hard-won rules (the first is now contested by the live database): - -- **Token intent: contested, do not trust either value blindly.** This page - long asserted that a bind token with `intent=app_password` is rejected by the - flow's password stage with `Invalid password` and only `intent=verification` - passes. The live token for `svc-jellyfin-ldap` is `app_password` - (`authentik_core_token`, 2026-10-18), so either the rule is wrong or the - current Jellyfin bind path is not that token. Nothing observable settles it - today: the LDAP outpost's retained log holds only its 5-minute - `initialized direct binder` / `Update providers` config cycles and no bind - activity *(unverified 2026-10-18: no bind was performed — reproducing it - requires the bind secret)*. Re-verify with a real bind before changing the - token's intent either way. -- **Search needs `search_full_directory`.** Without the RBAC grant the - outpost returns only the bind user itself — a `(memberOf=…)` filter then - matches zero entries and the plugin logs `Found no users matching X`. - The permission is object-level on the provider; grant it via an RBAC role - and retest. -- `ak shell` import path for the provider model in 2026.8: - `from authentik.providers.ldap.models import LDAPProvider`. - *(unverified 2026-10-18: importing it outside a configured Django shell fails - with `ImproperlyConfigured`, and no `ak shell` was driven against the live - auth service for this check)* +The objects behind it — LDAP provider, bind DN, service account and bind token, +RBAC grant, groups, application slug — and the hard-won bind rules are in +[[Authentik Outpost Reference]]. ## Configuration (secrets) @@ -340,7 +204,8 @@ every outpost session (see the ORM playbook). Applications** → add an application + a **Proxy** provider whose `external_host` is exactly the browser-facing origin (`https://`), `internal_host = http://caddy:8017`, and **all three** - flows set (see the provisioning pitfall above). Attach the provider to the + flows set (see the provisioning rules in + [[Authentik Outpost Reference]]). Attach the provider to the embedded outpost. 4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create the app with the correct redirect URI, and paste the `client_id` / @@ -373,47 +238,6 @@ every outpost session (see the ORM playbook). the LDAP section) — bumping only the tag would leave the LDAP outpost behind. Authentik migrates the schema on startup. -## Self-serve user-settings flow (password change) - -Users change their own account details through the `default-user-settings-flow` -(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the -target of the homepage footer's "account" link. It is a `stage_configuration` -designated flow, so it renders every bound `ConfigurableStage` for the logged-in -user. - -Stages, in execution order (the `FlowStageBinding.order` field, not the M2M -insertion order): - -| order | stage | type | what it does | -|---|---|---|---| -| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** | -| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user | - -**Password change is a prompt field, not a stage.** The `UserWriteStage` -writes whatever the preceding `PromptStage` collected: for the key `password` -it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix -is a `password` field on the `default-user-settings` prompt -(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a -`PasswordStage`. A `PasswordStage` is an *authentication* stage: its -`validate_password` runs `authenticate()` against the backends to check the -*current* password; it never sets a new one. (A `PasswordStage` was added on -2026-09-25 and removed the same day for exactly this reason.) - -Two ORM gotchas: `Flow.stages.add()` fails on this through model -(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is -MTI), so create the `FlowStageBinding` row directly with an explicit `order`; -and a prompt field is a `Prompt` row linked through `PromptStageField` -(`prompt_stage` + `prompt`; the order comes from `Prompt.order`). - -**A blank password must not set an empty password.** `set_password("")` would -make the account log in with no password, so a validation policy -(`default-user-settings-no-empty-password`, bound to the prompt via -`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]` -is `""` and allows it when set. Verified with the `PolicyEvaluator` -(empty → deny, set → allow, absent → allow) and end-to-end by driving -`UserWriteStageView.update_user` on a throwaway user (new password valid, old -invalid). - ## Outpost claim cache (`proxysession`) — 403 after a group add The embedded outpost does not re-evaluate group membership per request. Each @@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal (the inventory table carries both bare origins and trailing slashes; routing matches on hostname, verified 2026-09-02). `internal_host = http://caddy:8017`, all three flows set, and an - Application attached — the two provisioning rules above. + Application attached — the two provisioning rules in [[Authentik Outpost Reference]]. 2. **Route flip in ONE commit/deploy**: the vhost becomes `reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and inside the `http://:8017` listener add `@ host ` + @@ -521,22 +345,9 @@ EOF is no `authentik.policies.group` module in this build. *(unverified 2026-10-18: these module paths were not re-executed — importing outside a configured Django shell fails with `ImproperlyConfigured`)* -- For live DB truth prefer the postgres container, and it works without any - credential — the local socket is trusted, so a read-only audit is just - `podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -` - with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify - before guessing: 2026.x renamed both tables and columns, and the models are - MTI all the way down — `authentik_core_provider` holds `id`, `name` and the - three `*_flow_id` columns; the proxy table is - `authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column - really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2 - table in 2026.8), the LDAP table is - `authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`; - `authentik_core_application` attaches to a provider through `provider_id` - (its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost` - is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError` - is the same story from the ORM side: `type` lives in the subclass tables. - Never select secret columns (`client_secret`, `key`, `password`) into a log. +- The live database's table and column names (the 2026.x renames, the MTI parent + keys, the read-only `psql` audit recipe) are reference lookup: + [[Authentik Outpost Reference]]. ## User provisioning playbook (user-level account) diff --git a/serverAppConfigs-docs/Infrastructure/Grafana.md b/serverAppConfigs-docs/Infrastructure/Grafana.md index a1241ef..a7e6519 100644 --- a/serverAppConfigs-docs/Infrastructure/Grafana.md +++ b/serverAppConfigs-docs/Infrastructure/Grafana.md @@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026- client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths, Basic client auth, form-encoded token body, and the fork's userinfo-claims bug that leaves identity at the `sub` hash only) is in - [[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana: + [[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana: The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`, `ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so users land as Viewer with `login = `; no email/groups/role claims diff --git a/serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md b/serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md index ac73c83..83284d8 100644 --- a/serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md +++ b/serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md @@ -1,7 +1,7 @@ --- type: reference title: Homepage Status Probes -description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix). +description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions." tags: [homepage, status, probes, e2e, homepage-status] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Infrastructure/Homepage.md b/serverAppConfigs-docs/Infrastructure/Homepage.md index 8836544..243a56b 100644 --- a/serverAppConfigs-docs/Infrastructure/Homepage.md +++ b/serverAppConfigs-docs/Infrastructure/Homepage.md @@ -1,6 +1,6 @@ --- type: reference -description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation." +description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from." tags: - infrastructure - homepage diff --git a/serverAppConfigs-docs/Infrastructure/Observability Provisioning.md b/serverAppConfigs-docs/Infrastructure/Observability Provisioning.md index e19ec35..97e9823 100644 --- a/serverAppConfigs-docs/Infrastructure/Observability Provisioning.md +++ b/serverAppConfigs-docs/Infrastructure/Observability Provisioning.md @@ -1,7 +1,7 @@ --- type: reference title: Observability Provisioning -description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification. +description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands." tags: [observability, prometheus, grafana, provisioning, deployment] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Infrastructure/Observability.md b/serverAppConfigs-docs/Infrastructure/Observability.md index 2b11a80..41de35e 100644 --- a/serverAppConfigs-docs/Infrastructure/Observability.md +++ b/serverAppConfigs-docs/Infrastructure/Observability.md @@ -1,7 +1,7 @@ --- type: reference title: "Observability on Dankclaw" -description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured." +description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured." tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw] timestamp: 2026-09-11 --- diff --git a/serverAppConfigs-docs/Infrastructure/Prometheus.md b/serverAppConfigs-docs/Infrastructure/Prometheus.md index 31540e7..9b51a7a 100644 --- a/serverAppConfigs-docs/Infrastructure/Prometheus.md +++ b/serverAppConfigs-docs/Infrastructure/Prometheus.md @@ -1,6 +1,6 @@ --- type: reference -description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages." +description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural." tags: - infrastructure - prometheus diff --git a/serverAppConfigs-docs/Infrastructure/authentik-postgres.md b/serverAppConfigs-docs/Infrastructure/authentik-postgres.md deleted file mode 100644 index f0cd9d8..0000000 --- a/serverAppConfigs-docs/Infrastructure/authentik-postgres.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: reference -description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /" -tags: - - infrastructure - - authentik - - postgres -timestamp: 2026-08-28 ---- - -# authentik-postgres - -The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO / -OIDC provider. It is the only stateful store in the stack (users, flows, -sessions, events). Redis was removed in Authentik 2025.10; sessions and caching -now live here. - -- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`) -- **Image:** `docker.io/library/postgres:16-alpine` -- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes) -- **Health:** `pg_isready -d authentik -U authentik` - -See [Authentik.md](Authentik.md) for the full architecture, setup, backups -(`pg_dump`), and failure modes. diff --git a/serverAppConfigs-docs/Infrastructure/authentik-server.md b/serverAppConfigs-docs/Infrastructure/authentik-server.md deleted file mode 100644 index e7f8255..0000000 --- a/serverAppConfigs-docs/Infrastructure/authentik-server.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: reference -description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost." -tags: - - infrastructure - - authentik -timestamp: 2026-09-16 ---- - -# authentik-server - -The Authentik server — the core, the admin API, and the embedded Caddy outpost. -This is the service the Caddy `forward_auth` vhosts proxy to -(`authentik-server:9000`). It also serves the login page at -`https://auth.franksdank.cloud`. - -- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`) -- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`) -- **Port:** `9000` (published; the outpost + admin + login all live here) -- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker) -- **Health:** `http://localhost:9000/-/health/` - -See [Authentik.md](Authentik.md) for the full architecture, the Caddy -forward_auth pattern, first-time setup, and failure modes. diff --git a/serverAppConfigs-docs/Infrastructure/authentik-worker.md b/serverAppConfigs-docs/Infrastructure/authentik-worker.md deleted file mode 100644 index a771b29..0000000 --- a/serverAppConfigs-docs/Infrastructure/authentik-worker.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -type: reference -description: "The Authentik worker — background tasks: event processing, provider" -tags: - - infrastructure - - authentik -timestamp: 2026-09-16 ---- - -# authentik-worker - -The Authentik worker — background tasks: event processing, provider -synchronization, scheduled flows, and certificate renewal. It runs the same -image as the server; the `command: worker` selects the role. - -- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`) -- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`) -- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir) -- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server) -- **No port** — the worker has no HTTP endpoint. - -See [Authentik.md](Authentik.md) for the full architecture, setup, and failure -modes. diff --git a/serverAppConfigs-docs/LLM/Dankserver System Specs.md b/serverAppConfigs-docs/LLM/Dankserver System Specs.md index 53db7ff..682ffa3 100644 --- a/serverAppConfigs-docs/LLM/Dankserver System Specs.md +++ b/serverAppConfigs-docs/LLM/Dankserver System Specs.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: Dankserver System Specs -description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium. +description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture." tags: - dankserver - hardware diff --git a/serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md b/serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md index 5ab59d7..4eef7ca 100644 --- a/serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md +++ b/serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Dual R9700 Cooperation -description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed. +description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked." tags: - frank-medium - rocm diff --git a/serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md b/serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md index df516ab..e1a4edc 100644 --- a/serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md +++ b/serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: FastFlowLM NPU Inference -description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook. +description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook." tags: - fastflowlm - npu diff --git a/serverAppConfigs-docs/LLM/FreeToken Builds.md b/serverAppConfigs-docs/LLM/FreeToken Builds.md index 3120581..87caf65 100644 --- a/serverAppConfigs-docs/LLM/FreeToken Builds.md +++ b/serverAppConfigs-docs/LLM/FreeToken Builds.md @@ -1,7 +1,7 @@ --- type: reference title: FreeToken Builds -description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit. +description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit." tags: [freetoken, build, dankclaw, vllm, mtp] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/GGUF Compatibility.md b/serverAppConfigs-docs/LLM/GGUF Compatibility.md index aae7866..f66f42e 100644 --- a/serverAppConfigs-docs/LLM/GGUF Compatibility.md +++ b/serverAppConfigs-docs/LLM/GGUF Compatibility.md @@ -1,7 +1,7 @@ --- type: reference title: GGUF Compatibility and Runtime Lock-in -description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file. +description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can." tags: [gguf, llama.cpp, forks, quantization, speculative] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md b/serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md index 611dbad..9952905 100644 --- a/serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md +++ b/serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md @@ -1,7 +1,7 @@ --- type: reference title: Heterogeneous Split Triage -description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking. +description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys." tags: [heterogeneous, multi-device, llama-cpp, triage, pcie] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md b/serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md index 87385cd..1668a50 100644 --- a/serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md +++ b/serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md @@ -1,7 +1,7 @@ --- type: reference title: LLM Bench Windows and Measurement -description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding. +description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting." tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md b/serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md index 27a0e17..7ccd078 100644 --- a/serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md +++ b/serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md @@ -1,7 +1,7 @@ --- type: reference title: LLM Performance Diagnosis -description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing. +description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution." tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md b/serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md index f97e322..896f0ab 100644 --- a/serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md +++ b/serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md @@ -1,7 +1,7 @@ --- type: reference title: Llama Cpp Multi-Device Splits -description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage. +description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing." tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md b/serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md index 28c1c8a..f427e68 100644 --- a/serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md +++ b/serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md @@ -1,7 +1,7 @@ --- type: reference title: Llama Cpp Serving Sizing -description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes. +description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache." tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Model Swap and Rollback.md b/serverAppConfigs-docs/LLM/Model Swap and Rollback.md index 74c49bd..3b4fdad 100644 --- a/serverAppConfigs-docs/LLM/Model Swap and Rollback.md +++ b/serverAppConfigs-docs/LLM/Model Swap and Rollback.md @@ -1,7 +1,7 @@ --- type: reference title: Model Swap and Rollback -description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline. +description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert." tags: [models, swap, gguf, huggingface, rollback, deployment, verification] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/PCIe Link and Atomics.md b/serverAppConfigs-docs/LLM/PCIe Link and Atomics.md index 37dcfa3..7d4ce3c 100644 --- a/serverAppConfigs-docs/LLM/PCIe Link and Atomics.md +++ b/serverAppConfigs-docs/LLM/PCIe Link and Atomics.md @@ -1,7 +1,7 @@ --- type: reference title: PCIe Link and Atomics -description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all. +description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives." tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md index 9044dd1..36fc071 100644 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md @@ -1,11 +1,11 @@ --- type: reference title: Ethernet Bandwidth Math -description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers. +description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it." tags: - pipeline-parallelism - research -timestamp: 2026-08-17 +timestamp: 2026-10-19 --- # Bandwidth Analysis: Multi-Node LLM Inference over Ethernet @@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2 [fp16 = 2 bytes] - Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB` - With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok` -**Full example — 27B model, 2 PP stages (1 boundary), batch=1:** +**Worked examples (1 boundary = 2 PP stages):** ``` -1 boundary × 4096 × 2 bytes = 8,192 bytes/tok -``` - -**Example — same model, batch=64:** -``` -1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok -``` - -**Example — 36B model, 2 PP stages, batch=1:** -``` -1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok +27B, batch=1: 1 boundary × 4096 × 2 bytes = 8,192 bytes/tok = 8 KB +27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288 = 512 KB/tok +36B, batch=1: 1 boundary × 6144 × 2 bytes = 12,288 bytes/tok = 12 KB ``` ### 1.2 Tensor Parallelism (TP) — for comparison @@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2 [L all-reduces total | 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s | | 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s | -### 2.3 Feasibility matrix +### 2.3 Feasibility -Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%). - -**27B model — PP (1 boundary), batch=1:** - -| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) | -|---|---|---|---| -| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | -| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | -| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | - -**27B model — PP (1 boundary), batch=16:** - -| | 1 GbE | 2.5 GbE | 10 GbE | -|---|---|---|---| -| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | -| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | -| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | - -**27B model — PP (1 boundary), batch=64:** - -| | 1 GbE | 2.5 GbE | 10 GbE | -|---|---|---|---| -| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | -| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | -| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | - -**27B model — TP (32 layers), batch=1:** - -| | 1 GbE | 2.5 GbE | 10 GbE | -|---|---|---|---| -| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | -| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | -| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | - -**27B model — TP (32 layers), batch=16:** - -| | 1 GbE | 2.5 GbE | 10 GbE | -|---|---|---|---| -| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | -| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | -| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | - -**27B model — TP (32 layers), batch=64:** - -| | 1 GbE | 2.5 GbE | 10 GbE | -|---|---|---|---| -| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | -| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | -| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | +Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%, +**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is +feasible on every link class — even the heaviest, 27B at batch=64 and 100 +tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible +except batch=64: **27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE** @@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K **Petals [1]:** - Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth) - For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization -- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1] - Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput -- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference) -- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation) **llama.cpp RPC [2]:** - Model weights and KV cache distributed across nodes proportionally to memory @@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute ## Appendix: Full bandwidth formulae -### Pipeline Parallelism (generation, batch=1) +### Pipeline and Tensor Parallelism (generation) + +`N` is the batch size; `N = 1` is the batch=1 form. ``` -bytes_per_boundary_per_tok = hidden_size × 2 [fp16] -total_bytes_per_tok = num_boundaries × hidden_size × 2 +bytes_per_boundary_per_tok = hidden_size × N × 2 [fp16, PP boundary] +bytes_per_layer_allreduce = hidden_size × N × 2 [TP, one all-reduce per layer] +total_bytes_per_tok = num_boundaries × hidden_size × N × 2 [PP] +total_bytes_per_tok = num_layers × hidden_size × N × 2 [TP] bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000 -``` - -### Pipeline Parallelism (generation, batch=N) -``` -bytes_per_boundary_per_tok = hidden_size × N × 2 -total_bytes_per_tok = num_boundaries × hidden_size × N × 2 -``` - -### Tensor Parallelism (generation, batch=1) - -``` -bytes_per_layer_allreduce = hidden_size × 2 -total_bytes_per_tok = num_layers × hidden_size × 2 -bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000 -``` - -### Tensor Parallelism (generation, batch=N) - -``` -total_bytes_per_tok = num_layers × hidden_size × N × 2 -``` - -### With 8-bit quantization (like Petals) - -``` -# Simply divide all fp16 byte counts by 2 -bytes_per_boundary_per_tok = hidden_size × 1 [int8] +# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2 +bytes_per_boundary_per_tok = hidden_size × 1 [int8] ``` diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md index 7ebb709..4fc57e2 100644 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md @@ -1,7 +1,7 @@ --- type: reference title: Expected Performance -description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. +description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on." tags: - pipeline-parallelism - performance @@ -13,7 +13,7 @@ timestamp: 2026-08-14 This page estimates the speed of a 25-40B model split across two machines over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]] -and the measured numbers in [[Mixed-Vendor Experiments]]. +and the mixed-vendor measurements in [[Multi-Node Runtimes]]. ## The Bandwidth Ceiling Is Not the Limit @@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes. ## See Also - [[Ethernet Bandwidth Math]] -- [[Mixed-Vendor Experiments]] +- [[Multi-Node Runtimes]] - [[Sample Script Set]] diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Mixed-Vendor Experiments.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Mixed-Vendor Experiments.md deleted file mode 100644 index f04ce78..0000000 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Mixed-Vendor Experiments.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -type: reference -title: "Mixed-Vendor Experiments" -description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s." -tags: - - pipeline-parallelism - - research -timestamp: 2026-08-14 ---- diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Multi-Node Runtimes.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Multi-Node Runtimes.md new file mode 100644 index 0000000..b99a4e6 --- /dev/null +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Multi-Node Runtimes.md @@ -0,0 +1,257 @@ +--- +type: reference +title: Multi-Node Runtimes +description: "How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts." +tags: + - pipeline-parallelism + - llamacpp + - vllm + - speculative +timestamp: 2026-10-19 +--- + +# Multi-Node Runtimes + +Two runtimes can put one model on two machines: llama.cpp over the ggml-rpc +backend, and vLLM with pipeline parallelism. This page owns the cross-runtime +facts — how each is built and launched, what each can and cannot be combined +with, and the mixed-vendor verdicts. Facts owned by a sibling page are linked, +not restated. Bytes-per-token arithmetic and link budgets belong to +[[Ethernet Bandwidth Math]], throughput estimates to [[Expected Performance]], +the real deployment and its verified gotchas to [[Qwen3.8 PP Deployment]], the +educational scripts to [[Sample Script Set]], drafter mechanisms to +[[DFlash DSpark Speculative Decoding]], single-host multi-GPU placement to +[[Llama Cpp Multi-Device Splits]]. + +## What each runtime is + +Both distribute by **layers**, not inside layers. + +| | llama.cpp over ggml-rpc | vLLM pipeline parallelism | +| --- | --- | --- | +| unit of distribution | a contiguous slice of layers (`--split-mode layer`, the default) | a contiguous slice of layers (`--pipeline-parallel-size`) | +| process model | one `llama-server`/`llama-cli` on the head; a `ggml-rpc-server` per worker | one `vllm serve` per node, one engine across them | +| what crosses the wire | whatever graph work the head offloads to the remote devices | the hidden state at each stage boundary, point to point | +| KV cache | distributed with the weights; the KV of a layer lives with the owner of that layer | each stage holds the KV of the layers it owns | +| speculation together with the split | yes in llama.cpp (see below) | no, per vLLM's own docs | + +Upstream's own summary is why layer-wise is the ethernet choice: pipeline +parallel "minimizes data transfers between GPUs but requires many tokens to +scale well", while tensor parallel "does multiple cross-GPU reductions per +layer … much more bottlenecked by the GPU interconnect speed". + +Do not carry the tensor path across machines: `--split-mode tensor` is +experimental, needs flash attention, rejects quantized KV, and is not +implemented for the MoE and hybrid architectures the guide lists — and +per-tensor placement across vendors is not expressible (§1 of [[Llama Cpp Multi-Device Splits]]). + +## llama.cpp over ggml-rpc + +### Build and launch + +Both sides take `-DGGML_RPC=ON` next to that host's own backend: + +```bash +cmake .. -DGGML_CUDA=ON -DGGML_RPC=ON # a CUDA worker; -DGGML_HIP=ON for an AMD one +cmake --build . --config Release +``` + +Worker side — `ggml-rpc-server` exposes ggml devices on the remote host: + +```bash +bin/ggml-rpc-server # endpoint 127.0.0.1:50052 by default +bin/ggml-rpc-server --device CUDA0 -c # or CUDA_VISIBLE_DEVICES=0; -c = local cache +``` + +- It exposes every accelerator it can see; a host with no accelerator exposes a + single `CPU` device. `--device` and `CUDA_VISIBLE_DEVICES` are equivalent + ways to narrow that set. +- `-c` keeps large tensors in `$HOME/.cache/llama.cpp/rpc` (redirect with + `LLAMA_CACHE`) so a big model does not re-stream per load; that cache + filling the worker disk is a recorded fleet incident ([[Qwen3.8 PP Deployment]]). + +Head side — build the local backends plus `-DGGML_RPC=ON`, then name the +workers: + +```bash +llama-server -m model.gguf --rpc 192.168.88.10:50052,192.168.88.11:50052 -ngl 99 +``` + +`--rpc` is a comma-separated list of `host:port`, one entry per +`ggml-rpc-server`; the head's own devices stay in the pool next to the remote +ones. Launch scripts for this shape: [[Sample Script Set]]. + +### Split, context, KV + +- Default placement: weights **and** KV across all devices, local and remote, + "in proportion to each device's available memory". Override with + `--tensor-split`, whose proportions follow the order in `--device`. Why the + fleet always pins it (a Strix reports a huge GTT window, so the automatic + split lands on the slow device) is [[Qwen3.8 PP Deployment]]'s. +- `-sm layer` (the default) is pipeline parallelism: each device holds a + contiguous slice of layers and the KV cache of a layer lives on the device + that owns that layer. +- KV types are the usual `-ctk` / `-ctv`, `f16` by default. `--fit` is on by + default and sizes unset arguments to device memory; turn it off to hold a + context you set on purpose. Context is one global `--ctx-size` — the RPC + README documents no per-node context or per-node KV knob. +- Upstream's out-of-memory ladder (stated for tensor mode, useful order + anyway): lower `--ctx-size`, then `--parallel`, then `--n-gpu-layers`. +- Transport is TCP; RDMA is negotiated at the handshake with no flag changes + and falls back to TCP unless both peers can use it (Linux RoCEv2 via + `libibverbs`, macOS RDMA over Thunderbolt 5 via `librdma`). It is point to + point — name the peer's RDMA-capable address in `--rpc` or the session stays + on TCP. `GGML_RPC_NO_RDMA=1` forces TCP; `GGML_RPC_DEBUG` traces the backend. + +Upstream calls the backend a proof of concept, "fragile and insecure" (see the +Security section). Two limits shape the design: decode is bound by the per-step +round trip, not by the link's bytes ([[Ethernet Bandwidth Math]]), and every +node must run the identical build or the handshake hangs ([[Qwen3.8 PP Deployment]]). + +## vLLM pipeline parallelism + +> **Provenance.** Flags and mechanics below were recorded 2026-08-14 and are +> not re-checked upstream in this pass (see "Unverified in this pass"); the +> incompatibility lines, the method catalog and the issue history are re-verified. + +- `--pipeline-parallel-size` divides the layers into contiguous stages; + `--tensor-parallel-size` still works inside a node, so the canonical two-node + shape is tp = GPUs per node, pp = nodes. +- Two backends: Ray (`--distributed-executor-backend ray`, the worker joining + a `ray start --head`) or multiprocessing (`--distributed-executor-backend mp` + with `--nnodes`, `--node-rank`, `--master-addr`, `--headless` on the + follower). `mp` avoids the Ray dependency; Ray brings the cluster manager. +- What the interconnect must carry: one activation tensor per stage boundary + per forward pass, point to point — not a per-layer reduction. That is the + whole cross-node traffic story; the byte arithmetic and link budgets are + [[Ethernet Bandwidth Math]]. Pin the NIC vLLM talks on + (`NCCL_SOCKET_IFNAME`) rather than letting it guess an interface. +- The PP bubble: with N stages and one sequence in flight, one stage works + while N-1 idle, so pp=2 costs half the machine at batch one. Batching is the + cure — pipeline parallelism "requires many tokens to scale well" — and the + microbatch formula is [[Ethernet Bandwidth Math]] §5.3. +- KV and context: each stage keeps the KV of its own layers, so the pool grows + with the stage count, and `--max-model-len` is not reduced by the split + (recorded 2026-08-14, unverified here). +- What it cannot combine with, in vLLM's words: "Pipeline parallelism is not + composable with speculative decoding as of `vllm<=0.15.0`", and "Speculative + decoding with a draft models is not supported in `vllm<=0.10.0`". Composing + the two has been requested since 2024 and closed stale (#6911, #10615); PP + also carries a Qwen3.5 bug report closed stale without a fix (#36643). +- Method catalog at v0.25.1: EAGLE, MTP, draft model, PARD, MLP, n-gram, suffix + decoding, hidden-state extraction, a custom proposer class, dynamic + speculative decoding; `dflash` is an accepted `method` value. + +## Speculative decoding across the split + +llama.cpp chains several implementations behind one comma-separated +`--spec-type` list; the split does not change the chain, only where its passes +run. Mechanisms, chaining laws and drafter KV sizing are +[[DFlash DSpark Speculative Decoding]]'s; only the split interaction is here. + +| Method | llama.cpp type | Over ggml-rpc | With vLLM PP | Why | +| --- | --- | --- | --- | --- | +| MTP | `draft-mtp` | works; not documented upstream, but run this way on this fleet's split head with acceptance measured ([[Qwen3.8 PP Deployment]]) | no | Context-backed drafter: it needs a draft context and runs in the head process, so RPC devices are simply more devices under it | +| DFlash | `draft-dflash` | same gate, same status as MTP | no | Same draft-context gate (`ctx_dft`); the drafter's KV lands on whichever device the draft context is built on — the one thing to check on a split | +| ngram-mod | `ngram-mod` | safest of the four: no draft model, no draft context, no remote draft pass | no | Draftless — proposals come from cached token n-grams in the head, so the split adds no device work | +| DSpark | `draft-dspark` | **unproven over RPC** — not a missing feature | no | `draft-dspark` exists upstream; nobody has measured it with the target split across two machines | + +**Correction (2026-10-19).** A retired stub in this directory asserted "DSpark +is vLLM-only." False. `draft-dspark` is in upstream llama.cpp — parsed in +`common/arg.cpp`, mapped to `COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK` in +`common/speculative.cpp`, where it carries a Markov head and an anchor-first +block layout and aborts with `DSpark draft has no confidence head: please set +--spec-draft-p-min 0` when the drafter has no confidence head. What is unproven +is running it over RPC, and that is the honest limit of the claim. + +The vLLM column is uniform because that statement is blanket: it names pipeline +parallelism, not a method, so n-gram and suffix are excluded as written. vLLM +gives no mechanism with it; "the drafter must sit next to the target's layers, +which a stage split separates" is a reading, not upstream's words. + +Upstream likewise documents neither for nor against `--rpc` plus +`--spec-type`. What the implementation does say is structural: every +speculative implementation is built in the head process against the head's +device set, and draftless types need no draft context at all. Routing draft +passes to a remote worker, however, has never been tested. + +## Mixed-vendor reality + +What works today: + +- **One vendor per node.** RPC is backend-agnostic by construction: upstream's + own diagram has a host exposing CUDA0+CUDA1, a host exposing Metal, and a + host exposing CUDA0+CPU, all driven by one main host over TCP. Each node + builds its own backend plus `-DGGML_RPC=ON`; nothing speaks CUDA-to-ROCm. +- **Identical build tag on every node**, worker restarted before head. The + verified version-skew gotcha is [[Qwen3.8 PP Deployment]]'s. +- **Speculation on the head** while the split carries the target (table above). +- **RDMA when both peers have it**, auto-negotiated at the handshake; the size + of the gain is third-party and unverified here. + +Unproven or broken: + +- **One binary holding both CUDA and ROCm for a same-host layer split**: + reported to compile and then list only CUDA devices (issue #11506, + third-party, unverified). Vulkan is the reported single-binary cross-vendor + path, and single-host placement is [[Llama Cpp Multi-Device Splits]]'s, whose + verdict is that per-tensor placement across a vendor mix never works. +- **vLLM across a CUDA node and a ROCm node**: no known experiment; both ends + run the same backend (recorded 2026-08-14). AMD's own two-Halo clustering + guide is tensor parallelism over RCCL, not a mixed-vendor split. +- **DSpark over RPC**: unproven, see the correction above. +- **The exact NVIDIA-desktop + Strix-Halo-NUC pair has no published + benchmark**, and the third-party mixed-vendor figures it is estimated from + (Metal+CUDA over 10 GbE, two Strix Halos over USB4, a three-card CUDA+ROCm set + over 1 GbE, a Vulkan triple in one box, an AMD four-node Strix cluster) live on + [[Expected Performance]] and [[Ethernet Bandwidth Math]], each named with its + model and hardware. None was re-verified in this pass. + +## Security + +`ggml-rpc-server` has **no authentication and no encryption**. Upstream's own +warning: the backend is "in a proof-of-concept development stage. As such, the +functionality is fragile and insecure. Never run the RPC server on an open +network or in a sensitive environment!" It binds `127.0.0.1:50052` by default — +safe by accident. Bind a routable address and any host that can reach the port +can allocate memory there and supply the tensors it computes on. Trusted LAN +only, firewalled or tunnelled; never a public address. + +## Unverified in this pass + +Every item was first recorded 2026-08-14, in the research that preceded this +page. Nothing is silently deleted; none is backed by the sources cited below. + +- **RPC protocol version and handshake rule** — recorded as protocol `5.0.0` + with "major must match, server minor must be ≥ client". The README's own + banner prints `Starting RPC server v3.0.0`; the conflict is unresolved. +- **RPC internals** — a hard cap of 16 servers, synchronous graph compute, no + tensor copy between two RPC servers, workers loading tensors on first use, a + hash-keyed cache above roughly 10 MB, asserts on quantized tensors whose rows + are not 512-aligned, and `-rpc` on `llama-bench` (the README names only + `llama-cli` and `llama-server`). +- **vLLM flag surface** — `--pipeline-parallel-size`, `--tensor-parallel-size`, + `--distributed-executor-backend {ray,mp}`, `--nnodes`, `--node-rank`, + `--master-addr`, `--headless`, the `ray start --head` join, + `NCCL_SOCKET_IFNAME`. +- **vLLM mechanics** — one activation handoff per stage boundary per forward + pass, a worked ~512 MB handoff for a hidden-5120 model at batch 8 and 2048 + positions, `--max-model-len` unchanged by PP, ~50% idle at pp=2 batch one. +- **vLLM throughput verdicts** for a 27B across two nodes over gigabit-class + links; superseded as an estimate by [[Expected Performance]]. +- **Mixed-vendor claims** — a CUDA+HIP build listing only CUDA devices + (#11506), Vulkan as the universal cross-vendor backend, AMD's two-Halo + clustering guide being RCCL tensor parallelism, WiFi costing about ten times + wired, the slowest hop setting the decode pace, RDMA worth a few percent. +- **Draft over RPC** — that the draft model stays on the head, and that routing + draft passes to a remote worker has never been confirmed. + +## Citations + +- [llama.cpp RPC README](https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md), accessed 2026-10-19 — server usage and device exposure, `-DGGML_RPC=ON` on both sides, the `--rpc host:port` list, proportional weight+KV split and `--tensor-split`, the `-c` local cache, RDMA negotiation and `GGML_RPC_NO_RDMA`, the proof-of-concept and no-authentication warning. +- [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md), accessed 2026-10-19 — split modes (`-sm layer` = pipeline parallelism, KV with the layer owner), `-ts`/`-ngl`/`-ctk`/`-ctv`/`-fit`, the pipeline-versus-tensor summary, the tensor-mode limits. +- [vLLM speculative decoding docs, v0.25.1](https://docs.vllm.ai/en/v0.25.1/features/speculative_decoding/), accessed 2026-10-19 — the pipeline-parallelism incompatibility statement, the draft-model version floor, the method catalog. +- [llama.cpp `common/speculative.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/common/speculative.cpp), accessed 2026-10-19 — the speculative implementation set including `draft-dspark`, its draft-context gate, its confidence-head error. +- vLLM issue #6911, "[Feature]: Combine pipeline parallelism with speculative decoding", opened 2024-07-29, closed not-planned 2025-03-28, accessed 2026-10-19. +- vLLM issue #10615, "[Usage]: Does speculative decoding support pipeline parallelism?", opened 2024-11-25, closed not-planned 2025-04-18, accessed 2026-10-19. +- vLLM issue #36643, "[Bug]: Qwen3.5 does not work with pipeline parallelism", opened 2026-03-10, closed not-planned 2026-09-24, accessed 2026-10-19. diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.6 27B Spec Methods.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.6 27B Spec Methods.md deleted file mode 100644 index d991baf..0000000 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.6 27B Spec Methods.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -type: reference -title: "Qwen3.6 27B Spec Methods" -description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math." -tags: - - pipeline-parallelism - - research -timestamp: 2026-08-14 ---- diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md index 4fb30f6..47133b7 100644 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md @@ -1,6 +1,6 @@ --- type: reference -description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX" +description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas." tags: - llm - pipeline diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md index 3fc7c9f..24313f6 100644 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md @@ -1,7 +1,7 @@ --- type: reference title: Sample Script Set -description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy. +description: "An educational two-node RPC split script set, for reading only — never run it or deploy it." tags: - pipeline-parallelism - sample @@ -255,7 +255,7 @@ is 30 GiB. ## See Also -- [[llama.cpp RPC]] +- [[Multi-Node Runtimes]] - [[Expected Performance]] - [[Ethernet Bandwidth Math]] -- [[Qwen3.6 27B Spec Methods]] \ No newline at end of file +- [[DFlash DSpark Speculative Decoding]] \ No newline at end of file diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md index e3ccc16..9e7f00f 100644 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md +++ b/serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md @@ -28,28 +28,24 @@ graph LR end ``` -- [[llama.cpp RPC]] — the working, production-capable path for two heterogeneous nodes over ethernet. -- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or multiprocessing and does not compose with speculative decoding yet. ## What the Research Found -- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state +- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. (about 10 KiB per token for a 27B model). 1 GbE supports hundreds of tokens per second on paper; the real limit is latency and RPC overhead. -- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference works today via llama.cpp RPC with separate CUDA and ROCm builds. The exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with published numbers yet. -- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only. -- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split +- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE. ## Deployment -- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw +- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. (Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in `scripts/`, morning-of test runbook. Verified gotchas: no `--device` with `--rpc`, default split is proportional to reported free memory (pin 1,1), @@ -57,7 +53,7 @@ graph LR ## Sample Scripts -- [[Sample Script Set]] — a two-node llama.cpp RPC split of +- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it. Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for the largest context that fits. For educational reading only. Do not run it in production. @@ -68,3 +64,9 @@ graph LR - [[frank-zealot]] - [[Dankserver System Specs]] - [[Dankclaw System Specs]] + +## Pages + +| Page | Owns | +|---|---| +| [[Multi-Node Runtimes]] | How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts. | diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/llama.cpp RPC.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/llama.cpp RPC.md deleted file mode 100644 index 4b64d9b..0000000 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/llama.cpp RPC.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -type: reference -title: "llama.cpp RPC" -description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding." -tags: - - pipeline-parallelism - - research -timestamp: 2026-08-14 ---- diff --git a/serverAppConfigs-docs/LLM/Pipeline Parallelism/vLLM Pipeline Parallelism.md b/serverAppConfigs-docs/LLM/Pipeline Parallelism/vLLM Pipeline Parallelism.md deleted file mode 100644 index 0f4fd32..0000000 --- a/serverAppConfigs-docs/LLM/Pipeline Parallelism/vLLM Pipeline Parallelism.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -type: reference -title: "vLLM Pipeline Parallelism" -description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support." -tags: - - pipeline-parallelism - - research -timestamp: 2026-08-14 ---- diff --git a/serverAppConfigs-docs/LLM/Quantization Production.md b/serverAppConfigs-docs/LLM/Quantization Production.md index dae540b..92af878 100644 --- a/serverAppConfigs-docs/LLM/Quantization Production.md +++ b/serverAppConfigs-docs/LLM/Quantization Production.md @@ -1,3 +1,13 @@ +--- +type: "howto" +title: "Quantization Production" +description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file." +tags: + - llm + - quantization +timestamp: 2026-10-19 +--- + # Quantization Production — making our own quants, per card and runtime **Research page (2026-10-08).** Question: how do WE produce quantizations diff --git a/serverAppConfigs-docs/LLM/RDNA4 Container Serving.md b/serverAppConfigs-docs/LLM/RDNA4 Container Serving.md index d722203..57cc540 100644 --- a/serverAppConfigs-docs/LLM/RDNA4 Container Serving.md +++ b/serverAppConfigs-docs/LLM/RDNA4 Container Serving.md @@ -1,7 +1,7 @@ --- type: reference title: RDNA4 Container Serving -description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan. +description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage." tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md b/serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md index d98b8df..3633dd2 100644 --- a/serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md +++ b/serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md @@ -1,7 +1,7 @@ --- type: reference title: RDNA4 MoE Expert Offload VRAM Budget -description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder. +description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder." tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md b/serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md index fb00bc0..52c66b0 100644 --- a/serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md +++ b/serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md @@ -1,6 +1,6 @@ --- type: reference -description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series." +description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate." tags: - llm timestamp: 2026-09-03 diff --git a/serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md b/serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md index d393e97..8ab4837 100644 --- a/serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md +++ b/serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md @@ -1,6 +1,6 @@ --- type: reference -description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series." +description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape." tags: - llm timestamp: 2026-09-03 diff --git a/serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md b/serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md index 6bc2a45..6377ab3 100644 --- a/serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md +++ b/serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md @@ -1,7 +1,7 @@ --- type: reference title: Tcclaviger Fork Ops -description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence. +description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence." tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/VLLM Index.md b/serverAppConfigs-docs/LLM/VLLM Index.md index 69d687a..3275bdb 100644 --- a/serverAppConfigs-docs/LLM/VLLM Index.md +++ b/serverAppConfigs-docs/LLM/VLLM Index.md @@ -1,6 +1,6 @@ --- type: reference -description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw." +description: "Locator for every vLLM page in this vault." tags: - llm - vllm diff --git a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md index f9ffc23..32017c3 100644 --- a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md +++ b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18) -description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path. +description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line." tags: - dankclaw - iommu diff --git a/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md b/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md index 40339e4..31732b4 100644 --- a/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md +++ b/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Frontier MoE on Dankserver — what fits, on which runtime -description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded. +description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded." tags: - dankserver - r9700 diff --git a/serverAppConfigs-docs/LLM/dealign-ornith-crack.md b/serverAppConfigs-docs/LLM/dealign-ornith-crack.md index b3e618a..dc261c8 100644 --- a/serverAppConfigs-docs/LLM/dealign-ornith-crack.md +++ b/serverAppConfigs-docs/LLM/dealign-ornith-crack.md @@ -1,7 +1,7 @@ --- type: reference title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver -description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot. +description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card." tags: - llamacpp - cuda @@ -9,6 +9,7 @@ tags: - moe - ornith - mtp + - retired-endpoint timestamp: 2026-08-24 --- @@ -123,4 +124,4 @@ llama-server \ - [[deployment]] — deploy and update process - [[Dankserver System Specs]] — the box -- [[frank-redo-2026-08-17]] — fleet topology history \ No newline at end of file +- [[frank-redo-2026-08-17]] — fleet topology histor diff --git a/serverAppConfigs-docs/LLM/deployment.md b/serverAppConfigs-docs/LLM/deployment.md index 93f6d07..8e528b9 100644 --- a/serverAppConfigs-docs/LLM/deployment.md +++ b/serverAppConfigs-docs/LLM/deployment.md @@ -1,5 +1,5 @@ --- -type: guide +type: howto title: LLM Deployment description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). tags: diff --git a/serverAppConfigs-docs/LLM/frank-archon.md b/serverAppConfigs-docs/LLM/frank-archon.md index 8f01e08..d5856f2 100644 --- a/serverAppConfigs-docs/LLM/frank-archon.md +++ b/serverAppConfigs-docs/LLM/frank-archon.md @@ -1,7 +1,7 @@ --- type: reference title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)" -description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks." +description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught." tags: - llm - frank-archon @@ -11,6 +11,7 @@ tags: - r9700 - tp2 - expert-offload + - endpoint timestamp: 2026-10-18 --- @@ -139,8 +140,10 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti | 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy | | 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet | -Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]] -(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here. +Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era +pages are **not** superseded here: [[frank-medium]] is the locator for that +endpoint's whole shape history, including the unresolved end date of the +vllm-radiance era (this table opens on 09-26; the era page claims 10-10). ## Laws learned here @@ -438,3 +441,4 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]], [[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]], [[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]], [[frank-medium-radiance-mxfp4]], [[LLM/index]]. +--- diff --git a/serverAppConfigs-docs/LLM/frank-dragoon.md b/serverAppConfigs-docs/LLM/frank-dragoon.md index e06fa33..1312bac 100644 --- a/serverAppConfigs-docs/LLM/frank-dragoon.md +++ b/serverAppConfigs-docs/LLM/frank-dragoon.md @@ -1,7 +1,7 @@ --- type: reference title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)" -description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it." +description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it." tags: - llm - frank-dragoon @@ -9,6 +9,7 @@ tags: - strix-halo - gfx1151 - halogen + - endpoint timestamp: 2026-10-18 --- @@ -234,3 +235,4 @@ created it — earlier topologies in [[frank-redo-2026-08-17]]. - Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] · [[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] · [[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]]. +--- diff --git a/serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md b/serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md index 63b726e..706ed55 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md +++ b/serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md @@ -2,7 +2,7 @@ timestamp: 2026-09-17 type: reference title: frank-medium — Batch and Ubatch Sweep -description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides. +description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced." tags: - benchmark - llamacpp @@ -11,6 +11,7 @@ tags: - dankserver - layer-split --- + - retired-endpoint # frank-medium — Batch and Ubatch Sweep @@ -222,4 +223,5 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating. difference: with no KV to attend over, per-pass overhead dominates and a bigger micro-batch wins; at 64k of resident KV it does not. - [[Dual-R9700-Cooperation]] — why the cards are layer-split at all -- [[Dankserver System Specs]] — hardware inventory +- [[Dankserver System Specs]] — hardware inventory--- +timestamp: 2026-09-17 diff --git a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md index 7909ff5..459ad6e 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: frank-medium llama.cpp Runbook (DFlash2 drafter) -description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj. +description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on." tags: - frank-medium - dankserver @@ -12,6 +12,7 @@ tags: - gfx1201 - r9700 - layer-split + - retired-endpoint timestamp: 2026-09-20 --- @@ -730,3 +731,4 @@ fleet. - [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512, why the logical batch is free, and the harness traps +--- diff --git a/serverAppConfigs-docs/LLM/frank-medium-model-swap.md b/serverAppConfigs-docs/LLM/frank-medium-model-swap.md index 1ab4642..4499c5c 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-model-swap.md +++ b/serverAppConfigs-docs/LLM/frank-medium-model-swap.md @@ -1,7 +1,7 @@ --- -type: runbook +type: howto title: frank-medium Model Swap -description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path. +description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path." tags: - frank-medium - dankserver @@ -9,6 +9,7 @@ tags: - vllm-radiance - huggingface - runbook + - retired-endpoint timestamp: 2026-09-20 --- @@ -155,9 +156,10 @@ back: 2. `deploy.sh dankserver --service frank-medium --skip-e2e`. 3. Re-run the capability battery. -The 2026-09-20 rollback target is the MXFP4 checkpoint at -`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single -safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`. +The 2026-09-20 rollback target is the MXFP4 checkpoint with +`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that +payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a +local file. ## 2026-09-20 swap record (MXFP4 → FP8) @@ -184,3 +186,4 @@ safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`. DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9 vs 19.4 GiB). +--- diff --git a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md index d1b99a4..54d27d9 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md +++ b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: frank-medium vllm-radiance MXFP6 Runbook -description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era. +description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload." tags: - frank-medium - dankserver @@ -14,6 +14,7 @@ tags: - rccl - pcie-atomics - quadlet + - retired-endpoint timestamp: 2026-09-21 --- @@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and `--gpu-memory-utilization 0.98`. The Swift variant is a finetune of `ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may -drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec -decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to -`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image -back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/ -vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to +drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B +below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the +rollback**, and as of the 2026-10-19 host census it is **no longer on disk**: +`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a +rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub +`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the +image back to `localhost/radiance-vllm:0.9.3` (re-pull +`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to 486,464 tokens with bf16 KV and held the full 262,144 window). **Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.** @@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn with growing history) shows what happens at that limit: turns alternate between **~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted and must be prefilled again, because two families per card plus their cached -prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover, -both replicas steady — and per-family pinning kept the fast turns on the replica -that holds the prefix. +prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast +turns on the replica that holds the prefix. Read that as a capacity budget, not a defect: two concurrent large-context sessions per card are comfortable, three or more of that size trade turns @@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here. the same image and kernel 7.0.0-30 failing identically. - `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table this page compares against. +--- diff --git a/serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md b/serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md index 53a2c4e..b370466 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md +++ b/serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md @@ -1,7 +1,7 @@ --- -type: configuration +type: reference title: frank-medium ROCmFPX Runbook -description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201). +description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required." tags: - frank-medium - dankserver @@ -10,6 +10,7 @@ tags: - mtp - gfx1201 - r9700 + - retired-endpoint timestamp: 2026-09-15 --- diff --git a/serverAppConfigs-docs/LLM/frank-medium.md b/serverAppConfigs-docs/LLM/frank-medium.md new file mode 100644 index 0000000..98a906e --- /dev/null +++ b/serverAppConfigs-docs/LLM/frank-medium.md @@ -0,0 +1,65 @@ +--- +type: reference +title: frank-medium (retired) — the Dankserver endpoint before frank-archon +description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree." +timestamp: 2026-10-19 +tags: + - llm + - dankserver + - frank-medium + - retired + - retired-endpoint +--- + +# frank-medium (retired) — the Dankserver endpoint before frank-archon + +`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live +endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every +shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon` +name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** — +which era page holds which payload's evidence — and carries no measurements: every +number stays in the era page that measured it. + +## Eras + +Windows are quoted from each era page's own banner; no date here is derived. + +| Window | Runtime | Payload | Shape | Page | Rollback | +|---|---|---|---|---|---| +| 2026-09-15 → 2026-09-16 | llama.cpp, ROCmFPX fork pinned at commit `36e9acd` | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) | single card; the fork is the only loader | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork | +| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags | +| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape | +| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4, then MXFP6 (`paroquant_mxfp6`) with bf16 KV | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` | + +`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run +on the layer-split pair at its deployed shape, and its numbers belong to that +campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of +the 2026-09-20 swap procedure — while the procedure itself is owned by +[[Model Swap and Rollback]]. + +## Laws + +None restated. Each era page carries its own laws and links them to the technique +page that owns the mechanism ([[PCIe Link and Atomics]], +[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]], +[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]). + +## Open questions + +Two date claims in the era pages do not reconcile, and both are left standing rather +than silently corrected: + +- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the + next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The + MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which + one directly followed the ROCmFPX fork cannot be settled from the repo. +- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while + [[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded + fortnight on that endpoint or a stale banner; nothing else in the repo covers + 2026-09-26 → 10-10 for port 7770. + +## Files + +No repo files own this endpoint: its units, env and serve scripts were renamed to +`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds +retired serve scripts from these eras as reference only. diff --git a/serverAppConfigs-docs/LLM/frank-npu.md b/serverAppConfigs-docs/LLM/frank-npu.md index cfdce4a..fd7432b 100644 --- a/serverAppConfigs-docs/LLM/frank-npu.md +++ b/serverAppConfigs-docs/LLM/frank-npu.md @@ -1,7 +1,7 @@ --- type: reference title: frank-npu (retired) — FLM NPU endpoint on Dankclaw -description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went. +description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went." tags: - flm - fastflowlm @@ -10,6 +10,7 @@ tags: - dankclaw - frank-npu - retired + - retired-endpoint timestamp: 2026-09-11 --- diff --git a/serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md b/serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md index 687be05..0832dca 100644 --- a/serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md +++ b/serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md @@ -1,9 +1,10 @@ --- type: reference -description: "This page records the full redo of the inference fleet on 2026-08-17." +description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way." tags: - llm - frank + - retired-endpoint timestamp: 2026-09-03 --- @@ -223,3 +224,4 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17 evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is the main TTFT lever for pi, which re-sends the same system prompt + tool schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file. +--- diff --git a/serverAppConfigs-docs/LLM/frank-zealot.md b/serverAppConfigs-docs/LLM/frank-zealot.md index 2b87766..1dc103c 100644 --- a/serverAppConfigs-docs/LLM/frank-zealot.md +++ b/serverAppConfigs-docs/LLM/frank-zealot.md @@ -1,7 +1,7 @@ --- type: reference title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV" -description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files." +description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws." tags: - llm - frank-zealot @@ -9,6 +9,7 @@ tags: - llama-cpp - gguf - rtx3090 + - endpoint timestamp: 2026-10-18 --- diff --git a/serverAppConfigs-docs/LLM/freetoken-benchmark-sweep.md b/serverAppConfigs-docs/LLM/freetoken-benchmark-sweep.md index 2ab0249..df6eaa4 100644 --- a/serverAppConfigs-docs/LLM/freetoken-benchmark-sweep.md +++ b/serverAppConfigs-docs/LLM/freetoken-benchmark-sweep.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: FreeToken Benchmark Sweep — Dankserver RTX 3090 -description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. +description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime." tags: - freetoken - benchmark diff --git a/serverAppConfigs-docs/LLM/ikllama-ornith-dankserver.md b/serverAppConfigs-docs/LLM/ikllama-ornith-dankserver.md index e6abe58..3647ce2 100644 --- a/serverAppConfigs-docs/LLM/ikllama-ornith-dankserver.md +++ b/serverAppConfigs-docs/LLM/ikllama-ornith-dankserver.md @@ -1,7 +1,7 @@ --- type: reference title: ik_llama.cpp on Dankserver (Ornith Q8_0) -description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags. +description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required." tags: - ikllama - llamacpp @@ -11,7 +11,8 @@ tags: - ornith - kv-cache - hadamard -timestamp: 2026-08-26 + - retired-endpoint +timestamp: 2026-10-19 --- # ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK) @@ -259,4 +260,4 @@ spec / `--spec-type` flags. The model file is not changed either way. build flags incl. `GGML_CUDA_FA_ALL_QUANTS` and `GGML_IQK_FA_ALL_QUANTS` - [ik_llama.cpp serverAppConfigs-docs/parameters.md](https://github.com/ikawrakow/ik_llama.cpp/blob/main/docs/parameters.md) — q8_KV (PR 208), K/V Hadamard options, `-cuda graphs=0` workaround, and - `--split-mode graph` MoE note (PR 1080) \ No newline at end of file + `--split-mode graph` MoE note (PR 1080) diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index d8367ef..0f4ecf4 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -9,37 +9,33 @@ each of those owns exactly that one thing. The incidents behind the laws are in | Page | Owns | |---|---| -| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… | -| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… | -| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… | +| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. | +| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. | +| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. | ## Retired endpoints | Page | Owns | |---|---| -| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… | -| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… | -| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… | -| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… | -| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… | -| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… | -| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. | -| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… | -| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. | -| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the | +| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. | +| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. | +| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. | +| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. | +| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. | +| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. | +| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. | +| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. | +| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. | +| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. | ## Pipeline Parallelism (series) | Page | Owns | |---|---| -| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… | -| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. | -| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… | -| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… | -| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… | -| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX | -| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… | -| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… | +| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. | +| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. | +| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. | +| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. | ## Techniques, hardware, trials @@ -47,43 +43,48 @@ each of those owns exactly that one thing. The incidents behind the laws are in |---|---| | [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw | | [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. | -| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… | -| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… | -| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… | +| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. | +| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. | +| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. | | [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. | -| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… | +| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. | | [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). | | [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. | -| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… | -| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… | -| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… | -| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. | -| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… | -| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… | +| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. | +| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. | +| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. | +| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. | +| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. | +| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. | | [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. | -| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. | -| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… | -| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… | -| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… | -| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… | -| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… | -| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… | -| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… | +| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. | +| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. | +| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. | +| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. | +| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. | +| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. | +| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. | | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. | -| [[Quantization Production]] | — | -| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on | +| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. | +| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. | | [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. | -| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… | -| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… | -| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. | -| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. | +| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. | +| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. | +| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. | +| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. | | [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. | -| [[swift15-campaign-worklog]] | — | -| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… | -| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. | -| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… | -| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. | +| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. | +| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. | +| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. | +| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. | +| [[VLLM Index]] | Locator for every vLLM page in this vault. | | [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. | -| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… | -| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… | -| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… | +| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. | +| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. | +| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. | + +## Pages + +| Page | Owns | +|---|---| +| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. | diff --git a/serverAppConfigs-docs/LLM/kingjones-q8agent-rocmfpx.md b/serverAppConfigs-docs/LLM/kingjones-q8agent-rocmfpx.md index 729375c..628c91e 100644 --- a/serverAppConfigs-docs/LLM/kingjones-q8agent-rocmfpx.md +++ b/serverAppConfigs-docs/LLM/kingjones-q8agent-rocmfpx.md @@ -1,10 +1,11 @@ --- type: reference -description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw." +description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it." tags: - llm - rocm - frank + - retired-endpoint timestamp: 2026-09-15 --- @@ -239,3 +240,4 @@ reads `--host/--port/--temp` as CLI flags, never as env vars. `gl=45`. See the `set_rows` build bug section above. - Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42, retired the same day for the AGENT model). See [[frank-dragoon]]. +--- diff --git a/serverAppConfigs-docs/LLM/ling-3.0-flash-trial.md b/serverAppConfigs-docs/LLM/ling-3.0-flash-trial.md index b9db9dc..331ae85 100644 --- a/serverAppConfigs-docs/LLM/ling-3.0-flash-trial.md +++ b/serverAppConfigs-docs/LLM/ling-3.0-flash-trial.md @@ -1,9 +1,9 @@ --- type: reference -description: "`bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`." +description: "Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict." tags: - llm -timestamp: 2026-09-03 +timestamp: 2026-10-19 --- # Ling-3.0-flash Q6_K Trial on Dankclaw @@ -109,6 +109,13 @@ re-enable `frank-dragoon-ling` from the repo unit - The heap is effectively full (98.7 of 111.2 GiB). Do not raise the context much without checking `--list-devices` free MiB. +## The other path: vLLM-ROCm (never shipped) + +A deployment plan existed for `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` +(INT4 W4A16, ~62 GB) served by vLLM-ROCm on Dankclaw. It never shipped; the +llama.cpp Q6_K fork trial above is what actually ran. The decision table the +plan carried is in git history. + ## See Also - [[frank-dragoon]] diff --git a/serverAppConfigs-docs/LLM/ling-3.0-flash-vllm-plan.md b/serverAppConfigs-docs/LLM/ling-3.0-flash-vllm-plan.md deleted file mode 100644 index 53d8e0d..0000000 --- a/serverAppConfigs-docs/LLM/ling-3.0-flash-vllm-plan.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -type: guide -description: "This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on" -tags: - - llm - - vllm - - rocm -timestamp: 2026-09-03 ---- - -# Ling-3.0-flash on Dankclaw — vLLM-ROCm Deployment Plan (never shipped) - -This page was the plan for a vLLM-ROCm deployment of -`jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on -Dankclaw. The deployment never shipped. The actual Ling work used the -llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]]. - -The trial service (`frank-dragoon-ling`) was retired and disabled under the boot -policy on 2026-08-17, in the same redo that renamed `frank-dragoon` to the -Qwen3.8-27B PP head ([[frank-redo-2026-08-17]]). Two fleet generations later the -name belongs to the halogen flash-server endpoint ([[frank-dragoon]]). See git -history for the full decision table. diff --git a/serverAppConfigs-docs/LLM/qwen3.8-spec-matrix.md b/serverAppConfigs-docs/LLM/qwen3.8-spec-matrix.md index 68f70b0..359cb91 100644 --- a/serverAppConfigs-docs/LLM/qwen3.8-spec-matrix.md +++ b/serverAppConfigs-docs/LLM/qwen3.8-spec-matrix.md @@ -1,6 +1,6 @@ --- type: reference -description: "This page records the speculative-decoding benchmark history and the" +description: "The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out." tags: - llm - speculative diff --git a/serverAppConfigs-docs/LLM/qwen3.8-strix-optimizations.md b/serverAppConfigs-docs/LLM/qwen3.8-strix-optimizations.md index e058b37..fc67dbc 100644 --- a/serverAppConfigs-docs/LLM/qwen3.8-strix-optimizations.md +++ b/serverAppConfigs-docs/LLM/qwen3.8-strix-optimizations.md @@ -1,6 +1,6 @@ --- type: reference -description: "This page answers one question: **how do we make Qwen3.8-27B serve faster on" +description: "How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought." tags: - llm timestamp: 2026-09-03 diff --git a/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md index 1cb81cd..1042c26 100644 --- a/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md +++ b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md @@ -1,3 +1,13 @@ +--- +type: "reference" +title: "Swift1.5 Campaign Worklog" +description: "Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm." +tags: + - llm + - campaign +timestamp: 2026-10-19 +--- + # Swift1.5 w4dn8 Campaign — Work Log Campaign branch `swift15-radiance-w4dn8`. Goal: convert diff --git a/serverAppConfigs-docs/LLM/ubatch-benchmark-large.md b/serverAppConfigs-docs/LLM/ubatch-benchmark-large.md index b228e18..a1a0ed8 100644 --- a/serverAppConfigs-docs/LLM/ubatch-benchmark-large.md +++ b/serverAppConfigs-docs/LLM/ubatch-benchmark-large.md @@ -1,7 +1,7 @@ --- -type: benchmark plan +type: reference title: frank-dragoon ubatch sweep -description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. +description: "The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness." tags: - llama.cpp - rocm diff --git a/serverAppConfigs-docs/LLM/vLLM Boot Forensics and Config Interrogation.md b/serverAppConfigs-docs/LLM/vLLM Boot Forensics and Config Interrogation.md index 741ef0a..cb97a9c 100644 --- a/serverAppConfigs-docs/LLM/vLLM Boot Forensics and Config Interrogation.md +++ b/serverAppConfigs-docs/LLM/vLLM Boot Forensics and Config Interrogation.md @@ -1,7 +1,7 @@ --- type: reference title: vLLM Boot Forensics and Config Interrogation -description: Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone), classifying the failure, and interrogating a live endpoint's true config and client contracts. +description: "Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config." tags: [vllm, forensics, journalctl, podman, config, contracts] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/vLLM Porting and Compile Triage.md b/serverAppConfigs-docs/LLM/vLLM Porting and Compile Triage.md index b0eeddf..fab8306 100644 --- a/serverAppConfigs-docs/LLM/vLLM Porting and Compile Triage.md +++ b/serverAppConfigs-docs/LLM/vLLM Porting and Compile Triage.md @@ -1,7 +1,7 @@ --- type: reference title: vLLM Porting and Compile Triage -description: Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that survive worker spawn) and the torch.compile/dynamo break triage, including the EAGLE3/DFlash2 aux-hidden-state protocol port rules. +description: "Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage." tags: [vllm, porting, torch-compile, dynamo, eagle3, dflash, glm] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/LLM/vllm_research_findings.md b/serverAppConfigs-docs/LLM/vllm_research_findings.md index cf1963a..dc918d6 100644 --- a/serverAppConfigs-docs/LLM/vllm_research_findings.md +++ b/serverAppConfigs-docs/LLM/vllm_research_findings.md @@ -1,6 +1,6 @@ --- type: reference -description: "vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method dflash` and `--spec-model` flags." +description: "Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method." tags: - llm - vllm @@ -10,7 +10,17 @@ timestamp: 2026-08-28 # Research: vLLM Parameters for Specified Configuration ## Summary -vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method dflash` and `--spec-model` flags. KV cache dtype `q8_0` is **not supported** — only fp8 variants and turboquant options are available. vLLM does not have a dedicated MoE expert CPU offload flag; general CPU offloading via `--cpu-offload-gb` is the closest option. The `--max-num-seqs` flag is confirmed. Podman GPU access requires `--gpus all` with the NVIDIA container toolkit. +vLLM v0.26.0 (released 2026-07-25, the current stable when this survey was +gathered on 2026-08-28; PyPI shows v0.32.0 as the latest on 2026-10-19, so +re-check this page before acting on it) supports DFlash speculative decoding. +The preferred CLI is the `--speculative-config` JSON object; `--spec-method` +and `--spec-model` are the per-field aliases defined in +`vllm/engine/arg_utils.py`. KV cache dtype `q8_0` is **not supported** — only +fp8 variants and turboquant options are available. vLLM does not have a +dedicated MoE expert CPU offload flag; general CPU offloading via +`--cpu-offload-gb` is the closest option. The `--max-num-seqs` flag is +confirmed. Podman GPU access requires `--gpus all` with the NVIDIA container +toolkit. ## Findings diff --git a/serverAppConfigs-docs/LLM/x4-pair-experiment.md b/serverAppConfigs-docs/LLM/x4-pair-experiment.md index 72fc264..d54348d 100644 --- a/serverAppConfigs-docs/LLM/x4-pair-experiment.md +++ b/serverAppConfigs-docs/LLM/x4-pair-experiment.md @@ -1,7 +1,7 @@ --- type: reference title: The x4 pair — Flash-Next across the 3090 and the Strix Halo iGPU -description: What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the tensor inventory that decides any split, the single flag that kills per-tensor placement across devices (-ot), the two-device layer splits that DO work (pure-Vulkan and cross-vendor CUDA+Vulkan, +7 % decode at 64k depth), and the harness-parsing trap that made the first round of measurements meaningless. +description: "Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement." tags: - llm - experiment diff --git a/serverAppConfigs-docs/Operations/Container Host Reachability.md b/serverAppConfigs-docs/Operations/Container Host Reachability.md index edda0aa..497c495 100644 --- a/serverAppConfigs-docs/Operations/Container Host Reachability.md +++ b/serverAppConfigs-docs/Operations/Container Host Reachability.md @@ -1,7 +1,7 @@ --- type: reference title: Container Host Reachability -description: The two podman/netavark reachability quirks on the fleet — containers cannot dial unpublished host ports (LAN IP or loopback), and netns membership decides what "reachable" means. Probes that prove it, fixes, and the pasta port gate. +description: "Two reachability quirks: containers cannot dial unpublished host ports, and netns membership decides reachability." tags: [podman, netavark, networking, rootless, reachability, caddy] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/Fleet Audit and Posture.md b/serverAppConfigs-docs/Operations/Fleet Audit and Posture.md index 170cb3e..3f8339a 100644 --- a/serverAppConfigs-docs/Operations/Fleet Audit and Posture.md +++ b/serverAppConfigs-docs/Operations/Fleet Audit and Posture.md @@ -1,7 +1,7 @@ --- type: reference title: Fleet Audit and Posture -description: Three standing audits — monitoring completeness (what should exist vs what is merely down), vhost proxy health decoding, and the behavioral security posture audit; plus the stash supersession check before dropping old work. +description: "Three standing audits: monitoring completeness, vhost proxy health, and behavioural security posture." tags: [audit, security, prometheus, caddy, monitoring, git] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/Fleet Ops Playbook.md b/serverAppConfigs-docs/Operations/Fleet Ops Playbook.md index ce7044a..94448b8 100644 --- a/serverAppConfigs-docs/Operations/Fleet Ops Playbook.md +++ b/serverAppConfigs-docs/Operations/Fleet Ops Playbook.md @@ -1,7 +1,7 @@ --- type: reference title: Fleet Ops Playbook -description: Incident-shaped fleet procedures — LLM endpoint down with an abandoned GPU holder, scratch vLLM boot windows (stop-first law), iterating on a fleet service before commit, incident-window reconstruction, and waking a sleeping host. +description: "Incident-shaped procedures: endpoint down with a GPU holder, stop-first bench windows, service iteration, waking a host." tags: [operations, incidents, gpu, systemd, wol, forensics] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md b/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md index a19a0e6..c81c57b 100644 --- a/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md +++ b/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md @@ -1,7 +1,7 @@ --- type: reference title: Payload and Disk Reclaim -description: Standing procedure for disk pressure on a fleet host — measure all buckets first, classify against the runbook index, prove nothing live references the payload, stage to trash with fingerprints, purge only on approval; plus the purge-and-refresh protocol with fetch verification. +description: "Disk pressure procedure: measure every bucket, classify, prove nothing references the payload, stage, purge on approval." tags: [operations, disk, podman, payloads, huggingface, trash] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/Remote Job Runner.md b/serverAppConfigs-docs/Operations/Remote Job Runner.md index fe428df..1a21977 100644 --- a/serverAppConfigs-docs/Operations/Remote Job Runner.md +++ b/serverAppConfigs-docs/Operations/Remote Job Runner.md @@ -1,7 +1,7 @@ --- type: reference title: Remote Job Runner -description: Detached long-job execution on fleet hosts with sentinel files (job.sh) — why ssh tail-watchers fail structurally, the script, its invariants, verification, and reporting discipline. +description: "Detached long jobs with sentinel files: why tail-watchers fail structurally, the script, its invariants, verification." tags: [operations, ssh, jobs, sentinel, detached] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/SSH and Remote Access.md b/serverAppConfigs-docs/Operations/SSH and Remote Access.md index 1ec5583..ab7f76f 100644 --- a/serverAppConfigs-docs/Operations/SSH and Remote Access.md +++ b/serverAppConfigs-docs/Operations/SSH and Remote Access.md @@ -1,7 +1,7 @@ --- type: reference title: SSH and Remote Access -description: Passwordless SSH bootstrap without sshpass/expect, remote execution against fish login shells, and diagnosing a frozen GUI from an SSH session. +description: "Passwordless SSH bootstrap without sshpass, remote execution against fish login shells, and frozen-GUI triage." tags: [ssh, remote, fish, gui, diagnosis] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Operations/index.md b/serverAppConfigs-docs/Operations/index.md index ec58fa1..a77c91d 100644 --- a/serverAppConfigs-docs/Operations/index.md +++ b/serverAppConfigs-docs/Operations/index.md @@ -1,28 +1,15 @@ ---- -type: index -description: Operational procedures for the fleet — job runners, disk and payload reclaim, container reachability, audits, SSH craft, and incident playbooks. -tags: [operations, index] -timestamp: 2026-10-05T00:00:00-07:00 ---- - # Operations Procedures for operating the fleet. The LLM-specific pages live under [LLM](../LLM/index.md); infrastructure and architecture under the `../Infrastructure/` and `../Architecture/` folders. -- [[Remote Job Runner]] — detached long-job execution with sentinel files; - why ssh tail-watchers fail -- [[Payload and Disk Reclaim]] — measure-classify-prove-stage-ask disk - procedure, plus the purge-and-refresh protocol -- [[Container Host Reachability]] — netavark/pasta reachability quirks and - the pasta port gate -- [[Fleet Ops Playbook]] — endpoint-down GPU-holder reclaim, scratch-boot - stop-first, pre-commit iteration, incident-window reconstruction, WoL -- [[Fleet Audit and Posture]] — monitoring gaps, vhost probes, security - posture, whole-fleet audit, bulk redaction, stash audits -- [[SSH and Remote Access]] — passwordless bootstrap without sshpass, fish - login shells, GUI-hang diagnosis over ssh +- [[Remote Job Runner]] — Detached long jobs with sentinel files: why tail-watchers fail structurally, the script, its invariants, verification. +- [[Payload and Disk Reclaim]] — Disk pressure procedure: measure every bucket, classify, prove nothing references the payload, stage, purge on approval. +- [[Container Host Reachability]] — Two reachability quirks: containers cannot dial unpublished host ports, and netns membership decides reachability. +- [[Fleet Ops Playbook]] — Incident-shaped procedures: endpoint down with a GPU holder, stop-first bench windows, service iteration, waking a host. +- [[Fleet Audit and Posture]] — Three standing audits: monitoring completeness, vhost proxy health, and behavioural security posture. +- [[SSH and Remote Access]] — Passwordless SSH bootstrap without sshpass, remote execution against fish login shells, and frozen-GUI triage. ## Related - [Observability Provisioning](../Infrastructure/Observability%20Provisioning.md) diff --git a/serverAppConfigs-docs/Research/AI Job Search and Resume Tools.md b/serverAppConfigs-docs/Research/AI Job Search and Resume Tools.md index cfdedc9..c3572e9 100644 --- a/serverAppConfigs-docs/Research/AI Job Search and Resume Tools.md +++ b/serverAppConfigs-docs/Research/AI Job Search and Resume Tools.md @@ -1,6 +1,6 @@ --- -type: research -description: "Self-hosted AI job-search and resume-formatting applications: the 2026 landscape, local-LLM fit, and a recommended homelab stack." +type: reference +description: "Self-hosted job-search and resume tools: the landscape, local-LLM fit, and the recommended homelab stack." tags: - research - job-search diff --git a/serverAppConfigs-docs/Research/ComfyUI ROCm and Vulkan.md b/serverAppConfigs-docs/Research/ComfyUI ROCm and Vulkan.md index 3abcb17..c33c0f8 100644 --- a/serverAppConfigs-docs/Research/ComfyUI ROCm and Vulkan.md +++ b/serverAppConfigs-docs/Research/ComfyUI ROCm and Vulkan.md @@ -1,6 +1,6 @@ --- type: reference -description: "Research into whether ComfyUI can use GPU acceleration — not just CPU — on the AMD Strix Halo APU (gfx1151) in Dankclaw." +description: "Whether ComfyUI can use the Strix Halo APU at all, and which backend path actually reaches the GPU." tags: - research - rocm diff --git a/serverAppConfigs-docs/Research/Llama Cpp Code Diff.md b/serverAppConfigs-docs/Research/Llama Cpp Code Diff.md index cfb62ca..3c0e73d 100644 --- a/serverAppConfigs-docs/Research/Llama Cpp Code Diff.md +++ b/serverAppConfigs-docs/Research/Llama Cpp Code Diff.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Llama.cpp Code Diff -description: Code-level diff of mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp with a focus on quantization code, ggml type numbers, and the differences that matter for Dankserver (RTX 3090) and Dankclaw (Strix Halo). +description: "Code-level diff of the three llama.cpp trees here: quantization code, type numbers, and what differs for our cards." tags: - llama.cpp - quantization diff --git a/serverAppConfigs-docs/Research/Llama Cpp Runtimes.md b/serverAppConfigs-docs/Research/Llama Cpp Runtimes.md index 054aad9..f3af259 100644 --- a/serverAppConfigs-docs/Research/Llama Cpp Runtimes.md +++ b/serverAppConfigs-docs/Research/Llama Cpp Runtimes.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Llama.cpp Runtimes -description: Compared mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp — positioning, quantization formats, code-level differences, and fit for Dankserver (RTX 3090) and Dankclaw (Strix Halo). +description: "The llama.cpp trees on this host compared: positioning, quantization formats, and fit for each machine." tags: - llama.cpp - quantization diff --git a/serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md b/serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md index bcb111a..5637fbe 100644 --- a/serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md +++ b/serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: Local TTS and Voice Cloning -description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment. +description: "Creating a local voice from sample files: the open-weight cloning landscape, the measured engine A/B, the audio stack." tags: - research - tts diff --git a/serverAppConfigs-docs/Research/Most Popular Self-Hosted AI Apps.md b/serverAppConfigs-docs/Research/Most Popular Self-Hosted AI Apps.md index 1610366..195bc73 100644 --- a/serverAppConfigs-docs/Research/Most Popular Self-Hosted AI Apps.md +++ b/serverAppConfigs-docs/Research/Most Popular Self-Hosted AI Apps.md @@ -1,6 +1,6 @@ --- -type: research -description: "The 15 most popular self-hosted AI applications of June–September 2026, ranked by GitHub stars, with homelab fit and adoption best practices." +type: reference +description: "The most-starred self-hosted AI applications of the year so far, with homelab fit and adoption notes." tags: - research - self-hosted diff --git a/serverAppConfigs-docs/Research/PICO-8 Asset Standards.md b/serverAppConfigs-docs/Research/PICO-8 Asset Standards.md index 161745b..ae67d02 100644 --- a/serverAppConfigs-docs/Research/PICO-8 Asset Standards.md +++ b/serverAppConfigs-docs/Research/PICO-8 Asset Standards.md @@ -1,7 +1,7 @@ --- type: reference title: PICO-8 Asset Standards -description: Production-grade standards for PICO-8 asset sheets — palette discipline, single perspective and light direction, detail flow and hierarchy, sheet organization, and the cohesion checklist. +description: "Asset-sheet standards for PICO-8: palette discipline, one light direction, detail hierarchy, cohesion checklist." tags: [pico8, game-dev, pixel-art, standards] timestamp: 2026-10-05T00:00:00-07:00 --- diff --git a/serverAppConfigs-docs/Research/SGLang.md b/serverAppConfigs-docs/Research/SGLang.md index 82adf06..102ecb3 100644 --- a/serverAppConfigs-docs/Research/SGLang.md +++ b/serverAppConfigs-docs/Research/SGLang.md @@ -1,7 +1,7 @@ --- -type: research +type: reference title: SGLang -description: What SGLang is, how it works (RadixAttention, paged KV, overlap scheduler, structured output), how it compares to llama.cpp and vLLM, and whether it fits Dankserver (RTX 3090) and Dankclaw (Strix Halo). +description: "What SGLang is, how it schedules and caches, how it compares to the runtimes here, and whether it fits this fleet." tags: - sglang - serving diff --git a/serverAppConfigs-docs/Research/Service Research Brief.md b/serverAppConfigs-docs/Research/Service Research Brief.md index b6075a4..d62aea6 100644 --- a/serverAppConfigs-docs/Research/Service Research Brief.md +++ b/serverAppConfigs-docs/Research/Service Research Brief.md @@ -1,6 +1,6 @@ --- type: reference -description: "Consolidated research from the `frank-zealot` research subagents (2026-08-14) plus the pi agent knowledge base." +description: "Consolidated upstream research on the fleet's LLM runtimes, dated to when it was gathered." tags: - research timestamp: 2026-09-03 diff --git a/serverAppConfigs-docs/Research/index.md b/serverAppConfigs-docs/Research/index.md index 9994974..560d980 100644 --- a/serverAppConfigs-docs/Research/index.md +++ b/serverAppConfigs-docs/Research/index.md @@ -5,32 +5,35 @@ findings doc; the project root index ([serverAppConfigs-docs/index.md](../index. ## LLM Inference -- [[Llama Cpp Runtimes]] — mainline vs charlie12345/ROCmFPX vs ik_llama.cpp: positioning, quantization, server fit -- [[Llama Cpp Code Diff]] — code-level quantization diff + comparison table + server recommendations -- [[SGLang]] — the structured-generation serving engine (llama.cpp/vLLM comparison, our- servers fit) +- [[Llama Cpp Runtimes]] — The llama.cpp trees on this host compared: positioning, quantization formats, and fit for each machine. +- [[Llama Cpp Code Diff]] — Code-level diff of the three llama.cpp trees here: quantization code, type numbers, and what differs for our cards. +- [[SGLang]] — What SGLang is, how it schedules and caches, how it compares to the runtimes here, and whether it fits this fleet. ## Image / Media -- [[ComfyUI ROCm and Vulkan]] — live-verified GPU path for ComfyUI on Strix Halo -- [[Lemonade and sd.cpp]] — the ggml-family image-generation alternative -- [[Jellyfin]] — media server research brief +- [[ComfyUI ROCm and Vulkan]] — Whether ComfyUI can use the Strix Halo APU at all, and which backend path actually reaches the GPU. +- [[Lemonade and sd.cpp]] — Why Lemonade Server can do image generation on Strix Halo when ComfyUI cannot. ## Audio / TTS -- [[Local TTS and Voice Cloning]] — creating a local voice from samples, the 2026 cloning landscape, the full audio stack, and a recommended homelab deployment +- [[Local TTS and Voice Cloning]] — Creating a local voice from sample files: the open-weight cloning landscape, the measured engine A/B, the audio stack. ## Career / Job Search -- [[AI Job Search and Resume Tools]] — self-hosted AI job-search + resume-formatting apps, local-LLM fit, recommended stack +- [[AI Job Search and Resume Tools]] — Self-hosted job-search and resume tools: the landscape, local-LLM fit, and the recommended homelab stack. ## Self-Hosted Landscape -- [[Most Popular Self-Hosted AI Apps]] — the 15 most popular self-hosted AI apps of Jun–Sep 2026, homelab fit, adoption best practices (HTML report: `popular-self-hosted-ai-apps.html`) +- [[Most Popular Self-Hosted AI Apps]] — The most-starred self-hosted AI applications of the year so far, with homelab fit and adoption notes. + +## Games / Creative + +- [[PICO-8 Asset Standards]] — Asset-sheet standards for PICO-8: palette discipline, one light direction, detail hierarchy, cohesion checklist. ## Backups -- [[Backblaze Integration]] — off-site backup plan (future) +- [[Backblaze Integration]] — Off-site backup plan for the bind-mounted data on both hosts. ## Legacy -- [[Service Research Brief]] — consolidated subagent research behind these pages +- [[Service Research Brief]] — Consolidated upstream research on the fleet's LLM runtimes, dated to when it was gathered. diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 9c77d48..439a818 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -11,125 +11,120 @@ is a locator: one row, one clause, no numbers. | Page | Owns | |---|---| -| [[Frank TTS]] | frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in… | -| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… | -| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… | -| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… | +| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. | +| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. | +| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. | ## Retired endpoints -These services no longer exist, so no live page owns them; rollback provenance -is inside each page. Every payload swap of a LIVE endpoint is an Eras row on -that endpoint's page, not a page. - | Page | Owns | |---|---| -| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… | -| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… | -| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… | -| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… | -| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… | -| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… | -| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. | -| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… | -| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. | -| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the | +| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. | +| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. | +| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. | +| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. | +| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. | +| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. | +| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. | +| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. | +| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. | +| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. | -## Dankclaw services (192.168.0.143) +## Dankclaw services | Page | Owns | |---|---| | [[ComfyUI]] | ComfyUI is a node-based AI image generation engine. | -| [[ComfyUI Qwen-Image Workflows]] | Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which… | +| [[ComfyUI Qwen-Image Workflows]] | Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next. | | [[Dankclaw Stack]] | Service inventory and routing overview for the Dankclaw AI, development, and observability host. | -| [[Frank TTS Engine]] | frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker,… | +| [[Frank TTS]] | The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role. | +| [[Frank TTS Engine]] | The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes. | | [[Gitea]] | Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service. | | [[Gluetun]] | Gluetun is a lightweight VPN client container. | | [[Hermes]] | Hermes is the self-improving AI agent from Nous Research. | | [[Hermes-Dashboard]] | The Hermes web dashboard is a browser UI for managing the agent. | -| [[Hermes-WebUI]] | The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH… | -| [[Marreta]] | The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet… | -| [[Open Notebook]] | open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out —… | +| [[Hermes-WebUI]] | The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN. | +| [[Marreta]] | The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain. | +| [[Open Notebook]] | The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house. | | [[OpenWebUI]] | OpenWebUI is a self-hosted chat interface for LLMs. | | [[Postgres]] | PostgreSQL is the database backend for Gitea. | | [[SearXNG]] | SearXNG is a privacy-respecting metasearch engine. | | [[SearXNG Providers]] | How Google and other upstream search engines get blocked behind the Gluetun | -| [[Seerr]] | Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived… | -| [[Swap Configuration]] | The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit.… | +| [[Seerr]] | The request manager for the *arr stack — the maintained successor to the archived Overseerr line. | +| [[Swap Configuration]] | The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat. | | [[Valkey]] | Valkey is the cache and limiter store for SearXNG. | -| [[Voice Packs]] | How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material… | +| [[Voice Packs]] | How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift. | -## Dankserver services (192.168.0.235) +## Dankserver services | Page | Owns | |---|---| | [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. | | [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. | | [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. | -| [[Dankserver Stack]] | Dankserver (192.168.0.235) runs the media and library stack. | +| [[Dankserver Stack]] | Service inventory and routing overview for the media and library host. | | [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at | -| [[Jellyfin]] | Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and… | -| [[mariadb]] | MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and… | +| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status. | +| [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. | | [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman | -| [[redis]] | Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup… | +| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. | | [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. | -| [[RomM-Scraping]] | This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping… | +| [[RomM-Scraping]] | Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library. | | [[Tautulli]] | Tautulli is the statistics, monitoring, and notification companion for | -## FrankBook-Pro (192.168.0.112) +## FrankBook-Pro | Page | Owns | |---|---| -| [[ARR-Stack]] | The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every… | +| [[ARR-Stack]] | The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas. | | [[lidarr]] | Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. | -| [[prowlarr]] | Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and… | -| [[qbittorrent]] | qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns,… | +| [[prowlarr]] | Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas. | +| [[qbittorrent]] | The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe. | | [[radarr]] | Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. | -| [[readarr]] | Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB… | +| [[readarr]] | The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client. | | [[sonarr]] | Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts. | ## Workstations | Page | Owns | |---|---| -| [[Noctalia and Niri]] | The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette… | +| [[Noctalia and Niri]] | The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning. | ## Cross-host infrastructure | Page | Owns | |---|---| -| [[Alertmanager]] | Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to… | +| [[Alertmanager]] | Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay. | | [[Alloy]] | Alloy is Grafana's log and telemetry shipper. | | [[Anubis]] | Anubis is the bot-challenge proxy that fronts the guarded public vhosts on | | [[Authentik]] | Authentik is the single sign-on (SSO) and OpenID Connect (OIDC) provider for the | -| [[authentik-postgres]] | The PostgreSQL 16 database for Authentik — the homelab's SSO / | -| [[authentik-server]] | The Authentik server — the core, the admin API, and the embedded Caddy outpost. | -| [[authentik-worker]] | The Authentik worker — background tasks: event processing, provider | +| [[Authentik Outpost Reference]] | Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO. | +| [[Authentik Processes]] | The three core Authentik containers — database, server and worker — with their units, images, roles and probes. | | [[Caddy]] | Caddy is the single reverse proxy for both hosts. | | [[FrankBook-Pro]] | FrankBook-Pro — the Mac mini running the *arr stack + qBittorrent inside a podman machine VM, gluetun-gated | | [[Grafana]] | Grafana visualizes the metrics that Prometheus collects from every service. | -| [[Homepage]] | A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link… | +| [[Homepage]] | The password-protected service list on Dankclaw: what each card shows and where its data comes from. | | [[Homepage Design Grilling]] | The decision log for the homepage design. | -| [[Homepage Status Probes]] | The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage… | +| [[Homepage Status Probes]] | How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions. | | [[LAN-DNS]] | The homelab gives every service a name under `franksdank.cloud`. | | [[Loki]] | Loki is the log aggregation backend of the observability stack. | | [[Node-Exporter]] | Node-Exporter exposes host metrics (CPU, RAM, disk, network) to Prometheus. | -| [[Observability]] | The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal… | -| [[Observability Provisioning]] | How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the… | +| [[Observability]] | The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured. | +| [[Observability Provisioning]] | Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands. | | [[Pi-hole]] | Pi-hole is the LAN DNS server for the homelab. | -| [[Prometheus]] | Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the… | +| [[Prometheus]] | Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural. | ## Repo architecture and policy | Page | Owns | |---|---| -| [[Container Network Namespaces]] | Which local services are addressable from which Podman network namespace on | -| [[Deploy State Triage]] | Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout,… | +| [[Container Network Namespaces]] | How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits. | +| [[Deploy State Triage]] | Reading deploy-state markers against live containers and deciding, safely, whether to redeploy. | | [[Deployment Flow]] | Service-scoped Git deployment for the two homelab hosts. | -| [[E2E Suite]] | `tests/e2e/` drives the **live fleet** through a real browser: fresh contexts | +| [[E2E Suite]] | The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional. | | [[Network Architecture]] | One Caddy fronts both hosts. | -| [[Quadlet Lifecycle]] | The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop… | -| [[Stale Signal Audit]] | The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers,… | +| [[Quadlet Lifecycle]] | The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval. | +| [[Stale Signal Audit]] | The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run. | | [[Storage Policy]] | All persistent data uses **bind mounts**: a host directory mapped directly into the container. | | [[Testing Policy]] | This repository tests its configuration at defined seams. | @@ -137,36 +132,46 @@ that endpoint's page, not a page. | Page | Owns | |---|---| -| [[Container Host Reachability]] | The two podman/netavark reachability quirks on the fleet — containers cannot dial unpublished host ports (LAN IP or… | -| [[Fleet Audit and Posture]] | Three standing audits — monitoring completeness (what should exist vs what is merely down), vhost proxy health… | -| [[Fleet Ops Playbook]] | Incident-shaped fleet procedures — LLM endpoint down with an abandoned GPU holder, scratch vLLM boot windows… | -| [[Payload and Disk Reclaim]] | Standing procedure for disk pressure on a fleet host — measure all buckets first, classify against the runbook index,… | -| [[Remote Job Runner]] | Detached long-job execution on fleet hosts with sentinel files (job.sh) — why ssh tail-watchers fail structurally,… | -| [[SSH and Remote Access]] | Passwordless SSH bootstrap without sshpass/expect, remote execution against fish login shells, and diagnosing a… | +| [[Container Host Reachability]] | Two reachability quirks: containers cannot dial unpublished host ports, and netns membership decides reachability. | +| [[Fleet Audit and Posture]] | Three standing audits: monitoring completeness, vhost proxy health, and behavioural security posture. | +| [[Fleet Ops Playbook]] | Incident-shaped procedures: endpoint down with a GPU holder, stop-first bench windows, service iteration, waking a host. | +| [[Payload and Disk Reclaim]] | Disk pressure procedure: measure every bucket, classify, prove nothing references the payload, stage, purge on approval. | +| [[Remote Job Runner]] | Detached long jobs with sentinel files: why tail-watchers fail structurally, the script, its invariants, verification. | +| [[SSH and Remote Access]] | Passwordless SSH bootstrap without sshpass, remote execution against fish login shells, and frozen-GUI triage. | ## Agent / harness | Page | Owns | |---|---| -| [[Agent Docs and Edits]] | Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline… | -| [[Command Center Dotfiles]] | The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite… | -| [[OMP Harness]] | The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via… | +| [[Agent Docs and Edits]] | Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each. | +| [[Command Center Dotfiles]] | How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas. | +| [[OMP Harness]] | The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits. | ## Research and methodology | Page | Owns | |---|---| -| [[AI Job Search and Resume Tools]] | Self-hosted AI job-search and resume-formatting applications: the 2026 landscape, local-LLM fit, and a recommended… | +| [[AI Job Search and Resume Tools]] | Self-hosted job-search and resume tools: the landscape, local-LLM fit, and the recommended homelab stack. | | [[Backblaze Integration]] | Off-site backup plan for the bind-mounted data on both hosts. | -| [[ComfyUI ROCm and Vulkan]] | Research into whether ComfyUI can use GPU acceleration — not just CPU — on the AMD Strix Halo APU (gfx1151) in… | +| [[ComfyUI ROCm and Vulkan]] | Whether ComfyUI can use the Strix Halo APU at all, and which backend path actually reaches the GPU. | | [[Lemonade and sd.cpp]] | Why Lemonade Server can do image generation on Strix Halo when ComfyUI cannot. | -| [[Llama Cpp Code Diff]] | Code-level diff of mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp with a focus on quantization code, ggml… | -| [[Llama Cpp Runtimes]] | Compared mainline llama.cpp, charlie12345/ROCmFPX, and ik_llama.cpp — positioning, quantization formats, code-level… | -| [[Local TTS and Voice Cloning]] | How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the… | -| [[Most Popular Self-Hosted AI Apps]] | The 15 most popular self-hosted AI applications of June–September 2026, ranked by GitHub stars, with homelab fit and… | -| [[PICO-8 Asset Standards]] | Production-grade standards for PICO-8 asset sheets — palette discipline, single perspective and light direction,… | -| [[Service Research Brief]] | Consolidated research from the `frank-zealot` research subagents (2026-08-14) plus the pi agent knowledge base. | -| [[SGLang]] | What SGLang is, how it works (RadixAttention, paged KV, overlap scheduler, structured output), how it compares to… | +| [[Llama Cpp Code Diff]] | Code-level diff of the three llama.cpp trees here: quantization code, type numbers, and what differs for our cards. | +| [[Llama Cpp Runtimes]] | The llama.cpp trees on this host compared: positioning, quantization formats, and fit for each machine. | +| [[Local TTS and Voice Cloning]] | Creating a local voice from sample files: the open-weight cloning landscape, the measured engine A/B, the audio stack. | +| [[Most Popular Self-Hosted AI Apps]] | The most-starred self-hosted AI applications of the year so far, with homelab fit and adoption notes. | +| [[PICO-8 Asset Standards]] | Asset-sheet standards for PICO-8: palette discipline, one light direction, detail hierarchy, cohesion checklist. | +| [[Service Research Brief]] | Consolidated upstream research on the fleet's LLM runtimes, dated to when it was gathered. | +| [[SGLang]] | What SGLang is, how it schedules and caches, how it compares to the runtimes here, and whether it fits this fleet. | + +## Pipeline Parallelism (series) + +| Page | Owns | +|---|---| +| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. | +| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. | +| [[Multi-Node Runtimes]] | How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts. | +| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. | +| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. | ## LLM technique and hardware references @@ -174,62 +179,51 @@ that endpoint's page, not a page. |---|---| | [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw | | [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. | -| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… | -| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… | -| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… | +| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. | +| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. | +| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. | | [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. | -| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… | | [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). | | [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. | -| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… | -| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… | -| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… | -| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. | -| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… | -| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… | +| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. | +| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. | +| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. | +| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. | +| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. | +| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. | +| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. | | [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. | -| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. | -| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… | -| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… | -| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… | -| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… | -| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… | -| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… | -| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… | +| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. | +| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. | +| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. | +| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. | +| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. | +| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. | +| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. | | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. | -| [[Quantization Production]] | — | -| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on | +| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. | +| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. | +| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. | | [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. | -| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… | -| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… | -| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. | -| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. | +| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. | +| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. | +| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. | +| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. | | [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. | -| [[swift15-campaign-worklog]] | — | -| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… | -| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. | -| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… | -| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. | +| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. | +| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. | +| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. | +| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. | +| [[VLLM Index]] | Locator for every vLLM page in this vault. | | [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. | -| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… | -| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… | -| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… | +| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. | +| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. | +| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. | -## Pipeline Parallelism (series) +## Change history | Page | Owns | |---|---| -| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… | -| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. | -| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… | -| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… | -| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… | -| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX | -| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… | -| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… | +| [[log]] | Chronological incident and law log for the whole bundle. | -## Uncategorised - -| Page | Owns | -|---|---| -| [[log]] | — | + diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 604b030..0a8d603 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -1,5 +1,52 @@ # Bundle Change Log +## 2026-10-19 +- **Thirteen vault pages were the same page twice, and the doubling was a + frontmatter tool, not a human.** `frank-archon`, `frank-dragoon`, + `frank-zealot`, `frank-npu`, `frank-redo-2026-08-17`, `kingjones-q8agent-rocmfpx` + and the five `frank-medium-*` era pages measured exactly 2× their `HEAD` size, + each with a second frontmatter block whose closing `---` was glued onto the end + of a body or timestamp line. **Law: a rewrite tool that re-reads the file it + just wrote reads its own output as the body, and a fence-repair pass keyed on + `line == "---"` never sees a glued fence** — the repair then inserts a fence in + the middle of the page and the next pass treats the tail as a new document. + Un-double rule that worked: cut where a `type:` line follows the first + frontmatter's close with more keys behind it, keep the copy whose body is a + superset, and prove it with the orphan census (`0` losses) before writing. +- **`.AGENTTRASH` is not vault content, and three tools forgot.** The link + checker resolved links *into* the trash (a live page pointing at a deleted page + read as healthy) and the docs seam demanded an index row for every trashed page. + Both now skip it, `check-okf.py` and `gen-vault-index.py` already did. +- **A cancelled agent's report is not evidence.** One subagent was killed by the + task runtime limit and its yield still described a 495-line owner page, a + verified era table and a byte-level census — the file did not exist, and the + table it quoted was self-referential. Another wrote its output to disk and then + died without reporting, leaving a compressed `log.md`. Both were caught only by + `wc`/`ls` on the file itself. **Law: verify the bytes, never the narrative** — + re-derive existence, line count and the orphan census after any delegated write, + whatever the yield claims. +- **`log.md` structure per OKF: one entry per date, bare `## YYYY-MM-DD`, newest + first.** 66 dated headings collapsed to 34 by merging same-date entries and + demoting their titles to bold leads; the file had been descending at the head + and ascending at the tail. Kept every line — an attempted prose compression was + rejected because the census showed it dropped laws held nowhere else (the + `qsa_sparse` 64-page-size override that refused 300,000 tokens, the + `{"status":"loading"}` health response that false-greened a deploy at six + seconds, the `libhiprtc.so` dev symlink, the stale `~/.docker/config.json` that + poisons podman's anonymous pulls). +- **Two date claims about the vllm-radiance era on the Dankserver port do not + reconcile, and neither was overwritten.** [[frank-archon]]'s Eras table opens on + 2026-09-26; [[frank-medium-radiance-mxfp4]] dates its own retired era to + 2026-10-10; nothing in the repo covers 09-26 → 10-10 for port 7770. The new + [[frank-medium]] locator records the dispute instead of picking a winner. +- **Host census settles a rollback claim.** `/home/dank/models/` on dankserver + holds `StillDeadcode`, `rad`, `tpl.jinja` — so the MXFP4 checkpoint that two + pages called the "on-disk rollback" is a re-fetch, now stated once, on the era + page that owns that payload. +- **Fleet state confirmed live, not from files**: frank-archon 3 × 200,000 fp8 KV, + frank-dragoon 2 slots × 200,704, frank-zealot 2 × 262,144; all three answer + `/health` 200 and every env value matches its page. + ## 2026-10-18 - **frank-zealot resliced to two windows of exact KV (Frank's call, PR #21).** `--ctx-size 786432 --parallel 3` at q8_0 K+V → **`524288 / 2` at f16 K+V**, so @@ -24,6 +71,729 @@ `/v1/chat/completions`. `AGENTS.md` said `/completion`; it now says the chat path. +**issue #3 research: AMD PerfOpt retires the reason for `amd_iommu=off`** + + +- Ask: does a recent kernel change remove the IOMMU translation penalty + that justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the + XDNA2 NPU come back? Answered and documented; **no host change** at the + time — the fix is not installable from the Ubuntu 26.04 archive. +- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13): merged + via the iommu tree 2026-09-24, first release **Linux 7.4**. It stops + GPA→SPA translation for the *privileged iGPU* only and leaves the IOMMU + fully armed for every other device, NPU included. Its only consumer + requires `AMD_IS_APU` **and the identity domain** — so PerfOpt rides on + top of `iommu=pt`, it does not replace it. +- **Three premises of the issue corrected.** (1) The 13–16 % is the + *vendor's* A/B against *their* previous boot line — our record copied the + figure on 2026-09-11 without ever measuring here. (2) The NPU does **not** + have to wait for 7.4: `iommu=pt` enables the IOMMU, which is all + `err=-19` asks for (`amdxdna` binds via `iommu_sva_bind_device()` into its + own SVA domain, independent of the default domain type). (3) The 460 + tok/s baseline is stale, so the window must measure `off` vs `pt` locally + and inherit no number. +- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and the + **Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % + on Strix Point / Krackan laptops) — the modest arm is the one that + describes this box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU + regression disables SVA outright. +- Self-correction inside the same session: the first pass claimed + `iommu=pt` alone already avoids the cost. It does not — an identity + attach keeps the page walk and enables ATS/PRI/PASID/GCR3, so `pt` + recovers **most** of the gap and PerfOpt is the last part of it. The + posted comment (Gitea issue #3) was edited to the corrected text. +- Artifacts: a new section in `dankclaw-iommu-translation-off.md` + (front-matter re-keyed to "vendor-claimed, never measured locally"), + `frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a + middle ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, and + a 4-step maintenance-window procedure that measures locally instead of + inheriting the vendor number. Nothing deployed — no service was touched. + +**boot line flipped: Dankclaw now boots `iommu=pt`** + + +- Window executed 01:24–01:35 on Frank's go-ahead, with the A/B the + research demanded. Provenance first: `/etc/default/grub` had never been in + the repo while the IOMMU line was the most consequential kernel decision + on the box, so it is now tracked at `dankclaw/host/grub` (fetched + byte-exact) and moves only through `scripts/set-dankclaw-bootline.sh` — + diff, install 0644 root:root, `update-grub`, **verify the default + `grub.cfg` entry**, reboot on `--reboot` only. The deployer plans nothing + under `dankclaw/host/`: kernel state is not a container restart. +- **Verdict: identity DMA costs nothing on this host.** Byte-identical + traffic, +2.0 % cold prefill going OFF→PT; the 13–16 % that justified + `amd_iommu=off` for five weeks never existed here. Decode's apparent + −9.7 % was instrument noise: three repeats of the same request on the + same boot spread 31.8–47.8 t/s. +- **The XDNA2 NPU is back:** `amdxdna` bound `0000:c6:00.1`, + `/dev/accel/accel0` `crw-rw---- root:render 261,0` with `dank` in + `render`. No endpoint deployed — revival needs a quadlet with + `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`. +- Deliberately NOT done: the `pp_dpm_fclk` level-7 pin (it changes iGPU + timing for every workload and nothing co-runs on the NPU yet) and any + PerfOpt work (kernel `7.0.0-38` has zero `perfopt` symbols; the 26.04 + archive caps at `7.0.0-38.38`). +- Gates after boot: 0 failed units, 32 IOMMU groups (31 identity + 1 DMA = + the Genesys Logic SD reader, a kernel quirk), interrupt remapping on; + frank-dragoon's ledger byte-for-byte the same shape and Gitea / Caddy + edge / Grafana answered 200. New tool `scripts/stage-frank-dragoon-eval.sh` + rebuilds the A/B corpus byte-identically — run it through `bash -lc`, `uv` + is in `~/.local/bin`. Docs re-keyed: the decision record + `dankclaw-iommu-translation-off.md` (new § *execution*), `frank-npu.md` + (precondition satisfied, endpoint still absent), AGENTS.md key lesson. + +**frank-zealot: Bonsai-2-9B PQ2_0 with the MTP head (3 × 262,144)** + +- 2026-10-18 — frank-zealot: 27B PQ2_0 → **Ternary-Bonsai-2-9B PQ2_0 with + the MTP head**, runtime moved to the PrismML vendor fork build **10777** + because the MTP arm needs the build that pairs MTP with a quantized V + cache. See [[frank-zealot]]. +- The win is the **window, not the quant**: every conversation goes back at + the trained 262,144 (the 27B era sold each slot 200,704), and the fleet's + widest context makes :7771 the **compaction hub** — `models.yml` + `contextWindow` 200704 → 262144. +- **MTP armed, then OFF by measurement.** The head is in the file and the + fork loads it, but the draft context bills its own **f16** KV over the + WHOLE pool, and at `UBATCH_SIZE=2048` that squeeze **halved decode**. The + documented MTP shape (`SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024`) was + declined — the decode gain is not worth spending the free card on it. The + arm is recorded in the env header, not left to rediscovery. +- Two bugs the trial caught, both fixed in the repo: + `${SPEC_PRESENCE_PENALTY:=0.0}` was a *default-assignment*, so the env + value always won and the documented "MTP runs without + `--presence-penalty`" arm was unreachable — now `${VAR=0.0}`, which lets + `SPEC_TYPE=draft-mtp` drop the flag. And `up{job="frank-zealot"}` had read + 0 for the **entire retained Prometheus history**: llama.cpp answers + **501** on `/metrics` unless `--metrics` is passed, so the scrape failed + beside a healthy `/health` and `LlmEndpointDown` fired for days. + `METRICS=1` is now the default and the seam pins it. +- Deployed 13:16 (`deploy.sh dankclaw --service frank-zealot`) — and the + first boot was **not the shipped payload**: a leftover + `frank-zealot.service.d/ptq1.conf` from the bench window pointed + `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit + booted that arm. `/health` 200, `/v1/models` echoed the alias, Playwright + 33/33 — **nothing in the gate reads the payload.** Drop-in moved to + `~/.AGENTTRASH/`, `daemon-reload`, restart: the shipped trio booted. (The + drop-in was not a rogue — see the PTQ1_0 entry below.) +- Measurement law: the first battery reported 117 and 72 t/s because it + timed natural generations that stopped at 24 tokens — wall-clock aggregate + on an early EOS reads 4× low. `ignore_eos` + the engine's own `timings` + numbers is the only honest way to time this endpoint. +- OpenWebUI `model` rows re-keyed in the live DB. Note for the next edit: + the values live inside the `params`/`meta` JSON columns — the table has no + `context_length`/`description` columns — and there is no `sqlite3` binary + on the host, so `python3` stdlib + `Connection.backup()`. + +**frank-zealot PTQ1_0 arm: measured, approved, and never written down** + +- **Correction to the entry above and to my own report.** The + `frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished + experiment. The session archive (`01a14c93`) carries Frank's own + instruction at **02:36**: replace frank-zealot with + `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8** mmproj, batch/ubatch + **3072/1536**, "fit as much context as possible in 2 contexts". +- It was the production payload **03:14:39 → 03:38:57** and served real + traffic; recovered from its own `print_timing` lines, PTQ1_0 ran + ≈47.6–49.0 t/s single-stream greedy decode (25–27 t/s per stream with two + streams live) and prefilled deep prompts faster than the PQ2_0 27B record + ever managed. +- Against the PQ2_0 27B record: PTQ1_0 freed the **1,201 MiB** its fetch + script promised and paid **−37 % decode** — the publisher's "dense + ternary unpacking loses on Ampere" warning, now confirmed on GA102. + Frank scratched the whole 27B plan at **03:33** ("scratch the current + frank-zealot plan completely… 9B… enable mtp… if you can possibly fit a + third context, do it"), which is what shipped at 13:16. +- The durable failure is mine: those numbers lived only in `/tmp` and the + journal, never in the repo, so the next agent on the host could not tell + an approved payload from a rogue drop-in and called it a hijack. The + fetch script's header now carries the measurement; + `/home/dank/models/Bonsai2-PTQ1_0/` stays on disk as the 27B rollback and + the drop-in stays trashed. + +**frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot** + + +- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33, and + the drop-in check the zealot incident bought ran FIRST: `DropInPaths` + empty, so the repo env governed. 2026-10-18 — frank-dragoon: the KV pool + doubled to 401,408 positions over the 200,704 ceiling; the measured boot + ledger owns the shape. See [[frank-dragoon]]. +- Warm reuse survived the raise: one **101,987**-token prompt took 81.5 s + cold and 0.3 s on repeats — `HALOGEN_CACHE_INPLACE=1` keeps the KV rows in + the pool, so a warm turn prefills nothing and the doubling evicted nothing + the cache needed. The RAM price is still real: host `MemAvailable` fell + **26 → 22 GiB**, because every pool position is RAM the n-gram table's + page cache loses. +- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the + engine reserves **11.0 GiB** and its preflight bills **11.3**. The measured + figure wins, and every "committed but not yet deployed" clause was retired + with the measurement (AGENTS.md, the env lore block, both index pages, the + iommu page, the runbook). +- PTQ1_0 provenance corrected (see the dedicated entry): the arm that + hijacked the zealot deploy was a **repo-tracked, Frank-ordered, measured** + candidate — `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, + and its header now carries the recovered numbers (≈48 t/s decode, 801 t/s + prefill at 141k) and the verdict: −37 % decode against PQ2_0 for 1,201 + MiB freed, then scratched by Frank's 03:33 order in favour of the 9B. + Nothing is owed on it. +- Census of the halogen HTTP surface while doing it, because the fleet bench + law quoted the wrong engine: frank-dragoon answers `/health`, `/metrics`, + `/v1/models`, `/v1/chat/completions`(POST) and 404s `/ping`, `/props`, + `/completion`, `/tokenize`; the chat response carries **no + `Server-Timing`** header. So the `/completion` `timings` object and the + `ttft;dur=` header are **llama.cpp** instruments — frank-zealot's, not + dragoon's. AGENTS.md's Benchmarking Policy re-keyed to say which engine + owns which instrument. + +**frank-tts phase 1: the cloned-voice endpoint, named and tested (not deployed)** + +- Voice-over goal, phase 1 (Gitea issue #2). New service **frank-tts** on + Dankclaw: `dankclaw/voiceservice/` (stdlib-only `app.py` + `Containerfile` + + rootless quadlet), `scripts/frank-tts.env`, `tests/frank_tts_unit.bats`. + One OpenAI-shaped `POST /v1/audio/speech` surface; `voice=` is + honoured only when `/voices//reference.wav` exists, otherwise a 400 + listing the voices that are — **no silent fallback voice**; oversized + input is a 414, not a half-narrated sentence; `/health` reports backend + + voice list; no audio ever reaches the log. +- **Naming law applied (Frank's call):** treat it as a first-class model + endpoint → alias-named `frank-tts` (TTS endpoint, so not `frank-asr`), + **port 7772**, joining `7770 frank-dragoon` / `7771 frank-zealot`; the + engine upstream keeps 8880 and is deliberately not a fleet endpoint. The + internal knob names stay `SHIM_*` (they name the façade component, not the + service). +- **Voice-pack law recorded** in the env header: a pack needs + `reference.wav` (24 kHz mono, 18–25 s) **and** the pinned engine's own + speaker artifact — required, beside the WAV, never replacing it — plus + `manifest.json` provenance whose offsets are always past the first 10 s of + the source (the source-material law). Enforcement is split honestly: the + shim gates on `reference.wav` (the only engine-agnostic term); the pack + builder is what will refuse to emit a pack without its artifact. +- Landed as **PR #4** (branch `wt/20261017-181502` → `main`, merged as + `49c4e19`) — the first PR on this repo through the API, which cost three + 422s to get right: the merge endpoint takes **`{"Do":"merge"}`**, `Do` + being a *string* naming the strategy — `DoMerge`/`merge_method` do not + exist on this Gitea, and swagger JSON is disabled on the instance, so the + 422 messages are the schema. Law saved to the `gitea-api-pr-operations` + skill. + +**TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion** + +- Frank listened to the four audition pairs (`qwentts-*.mp3` against their + `armB-*.mp3` mainline partners) and **passed every clone**. That deleted + the only axis the A/B could not measure, and the question was reframed: + **which engine is better for live output**. On that criterion RTF only has + to stay under 1.0 with margin — the discriminators are time-to-first-audio + and run-to-run spread, because an arm whose RTF can cross 1.0 stalls a + live stream mid-sentence. +- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, + 1.7 B talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on + 8880.** The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` + stays in the repo as the rollback. Three consequences are configuration, + not preference: boot-warm every voice, do not pin a cpuset (threads are + `hardware_concurrency()/2`), register every pack with its WAV (a lone + embedding is rejected). +- Two follow-ups deliberately **not** decided: the **0.6 B talker** (never + auditioned — every clip is 1.7 B) and **ICL vs embedding-only** + conditioning. Embedding-only is the shipped default because it is cheaper + and needs no `ref_text`, which this fleet cannot transcribe honestly; + `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice + opt-in for batch work where total time beats first byte. +- **Addressing law corrected** in [[Local TTS and Voice Cloning]]: the pair + may not meet at a host-loopback publish. netavark DNATs published ports + only and a bridge container cannot reach a host `127.0.0.1` bind — the + lesson that 502'd hermes — so `host.containers.internal:8880`, which + `scripts/frank-tts.env` documents, is correct only for a routable bind. + The deployed pair shares one quadlet network and the façade addresses the + engine by container name. +- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`) is **merged** as + `1eb8b21` (Frank's OK): open-notebook + SurrealDB quadlets on the private + bridge `10.89.4.0/24`, the API published on **5056** because seerr holds + `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT` moved 600 → **660** to sit + one round-trip outside frank-tts's own `SHIM_TIMEOUT`. Merged is not + runnable: nothing listens on 7772 until the voice stack lands, and the + episode destination stays interim pending Gitea #7. + +**open-notebook: the podcast generator's config lands (nothing deployed)** + + +- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — + notes and sources in, a multi-speaker episode out, script written by + **frank-zealot**, voices rendered by **frank-tts**, no cloud provider + anywhere in the path. Repo-side only: no host touched. Runbook: + [[Open Notebook]]. +- **Two containers, not one.** `lfnovo/open_notebook:v1-latest` plus + `surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` — + frank-dragoon's pinning style, because `:latest` moving underneath a + rootless service is how you get an unbootable pod after an unrelated + `systemctl-daemon-reload`. Both health probes are argument-only + (`curl -fsS …/health`, `/surreal is-ready`), and `is-ready` refuses to + pass without `SURREAL_PASS` in the environment — a credential that must + never ride the command line, so `Exec=` carries flags only. +- **Host 5055 was already spoken for** — seerr has held `127.0.0.1:5055` — + so the API publishes `0.0.0.0:5056:5055`, and because open-notebook's + frontend derives its API URL from the window origin when `API_URL` is + unset, `API_URL=http://192.168.0.143:5056` is set to match: a silent + remap would hang the UI on a port nobody listens on. 8502 publishes on + `0.0.0.0` because this service carries its own password (the + episode-audio route is unauthenticated); **8000 is published nowhere**, + where upstream exposes it on loopback with root credentials, and the seam + forbids it. +- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread + engine per request on a box that also serves two LLMs, while + open-notebook defaults to a 25-way self-DoS; both knobs are pinned to + `1` in `scripts/open-notebook.env`. The TTS clock goes to **660 — one + round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so a shim + give-up reads as the shim's error rather than the worker's; the seam + asserts the direction (`>`) against the frank-tts env instead of + restating a number. +- **What a credential needs in order to exist:** + `OPEN_NOTEBOOK_ENCRYPTION_KEY` must be set before the UI can save *any* + provider credential, and `OPEN_NOTEBOOK_PASSWORD` must stay **absent** to + remain auth-off — the app reads `os.environ.get(…) is not None`, so + `PASSWORD=` boots a login form asking for the empty string. The deployer + seeds `/home/dank/Containers/open-notebook/env` (600) from the host vault + and **refuses to start the pair while either secret is blank**, because a + half-configured open-notebook stores no credentials and fails quietly. +- **Episode destination is interim, by decision.** The destination is + **Gitea issue #7, `human-only`** — agents must not work it. Until Frank + settles it, the app keeps its own named volume and + `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` + (`DATA_FOLDER` is hard-coded `./data`), so finished MP3s are ordinary + files in his home directory instead of volume contents needing + `podman unshare`. One mount line; one deletion undoes it. +- **Two UI buttons will fail, and that is cosmetic:** Test Connection does + `GET /models` and the TTS probe synthesises voice `alloy`; + frank-tts serves `/health` and `/v1/audio/speech` only. Model rows are + created *by name* and the voice field is free text, so `narrator1/2/3` is + typed in and never discovered. Documented, not worked around. +- Three deployer seams cost time and are now law-shaped: a service must be + registered in **five** places — `SERVICE_ORDER`, `DANKCLAW_SERVICES`, the + `deploy-remote.sh` dispatch case, its pre-deploy requirement, and + `deploy.sh`'s dispatch regex, where an omission silently plans an empty + set for `--only`; a `dankclaw//*` case must precede the + `dankclaw/*.container` glob because that `*` crosses `/`; and the + changed-path routing lives *inside* the per-path loop, after the + `scripts/*.env)` case, not at the top of the file. + +**frank-tts-engine: the arm A engine becomes deployable (nothing deployed)** + +- Goal: turn the engine the day's A/B picked into a **repo-owned, + boot-persistent, self-healing container** — `frank-tts-engine`, + `qwentts.cpp` commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`, resident + `tts-server`, container port 8880, and **not** a fleet endpoint: the + endpoint stays `frank-tts` on 7772. Runbook: [[Frank TTS Engine]]. +- **No `PublishPort` at all, which the merged A/B states as law.** netavark + DNATs published ports only, so a `127.0.0.1:8880` publish is not a path to + a peer container (the trap that 502'd hermes) — while being the one + LAN-reachable surface of an unauthenticated synthesiser. Both voice + containers therefore sit on one committed quadlet network, + `frank-tts-engine.network` at **10.89.5.0/24** (the next free /24 after + open-notebook's 10.89.4.0/24), the façade addresses the engine by + container name, and diagnostics — including the deployer's health gate + (`wait_container_healthy`, there is no URL to dial) — run through + `podman exec`. +- **The boot warmer is a unit hook, not an operator habit.** The registry is + memory-only and the first request for a registered voice decodes the whole + reference, so `ExecStartPost=-/usr/bin/podman exec frank-tts-engine + /app/warm-voices.sh` with `TimeoutStartSec=900`; the `-` prefix is what + stops a failed warm from crash-looping the endpoint under + `Restart=always`, and it logs `cold=` and `warm=` per voice on every boot. +- **Thread law recorded before anyone tunes it away:** no `--threads` + exists; `src/backend.h` uses `hardware_concurrency()/2`, so a cpuset pin + silently changes the thread count. Nothing pins this container. +- Image builds from the pinned commit with the **proven** line, verbatim — + `GGML_BLAS=OFF` forced, every GPU backend off, no `-DGGML_NATIVE` because + ggml's default is `-march=native` for the building host: the image is + built where it runs. Weights fetched by + `scripts/download-frank-tts-engine-qwentts.sh` with byte and LFS-oid + checks; the deployer gates on `--verify-only` rather than pulling 1.4 GiB + inside a deploy. + +**frank-tts: the 7772 endpoint gets its deploy wiring, and voice naming is written down** + + +- Stage 2 of the voice stack: the **façade** in front of the engine is now a + deployable service too. Runbook: [[Frank TTS]]. +- **Two attached networks, in a meaningful order.** `dankclaw_default` + first — it owns the default route and is where consumers live + (open-notebook dials `http://frank-tts:7772/v1` by container name) — then + the voice pair's private bridge. `scripts/frank-tts.env`'s `TTS_UPSTREAM` + moves from `host.containers.internal:8880` to + `http://frank-tts-engine:8880`, the actual fix for a shape that could + never have worked: a bridge container cannot reach a host-loopback bind, + and the engine publishes nothing. +- **`Wants=` + `After=` on the engine, never `Requires=`:** the façade still + carries the `llama-tts-cli` rollback backend, which needs no upstream, so + an engine that will not boot must delay this endpoint, not take it down. +- The façade handler installs the **engine's** quadlet pair as well — not + tidiness: with a referenced `Network=` file absent the v1 Quadlet + generator produces no container unit at all — and it `mkdir -p + /home/dank/voices` as the deploy user, because rootless podman + auto-creates a missing bind source **root-owned** and the endpoint then + reports "no voices" instead of an error. +- **A voice is a lowercase directory name.** `^[a-z0-9][a-z0-9._-]*$`, + ≤ 64 chars, no slash/`..`/space. The shape is the intersection of two + rules that do not agree: `app.py` would accept mixed case, `tts-server` + lowercases names — so a pack named `Narrator1` resolves through the façade + and not through the engine, and the boot warmer refuses such a directory + loudly. `narrator1/2/3` are load-bearing for open-notebook. +- **No Prometheus job, on purpose.** The façade exports no `/metrics`, + Prometheus 3 refuses HTML/JSON content types (the `comfyui` and `services` + probes are documented down for exactly that), and `LlmEndpointDown` is + scoped `up{job=~"frank-.*"}` — so a `/health` scrape would page on Signal + every few minutes for a healthy endpoint. That is the frank-archon + never-green lesson pointed the other way, written in the page instead of + quietly ignored. + +**voice-pack builder: the 10-second source-material gate becomes code** + + +- Third slice of the voice stack: `scripts/build-voice-pack.sh` turns a + source recording plus a crop offset into a deployable pack — + `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the engine's own + `reference.spk` + `reference.rvq` made from it, and `manifest.json` + provenance. Tracked tunables in `scripts/voice-packs.env`; new runbook + [[Voice Packs]]. +- **The gate is the product.** The project-wide invariant — no voice + material from the first 10 s of a sample file — had only ever been a + sentence in `scripts/frank-tts.env`. An offset inside it now exits 1 + before a directory is created, and the same assertion runs again against + the manifest that was just written, next to the recorded sha256, so a pack + cannot carry a forbidden crop even if the env file is edited into + nonsense. `10` stays a constant in the builder; the env file carries + offsets, never the law. +- **The extractor's contract was read, not guessed:** `qwen-codec` in the + pinned qwentts.cpp commit has **no `-o`** and names both artifacts next to + the input with the extension swapped. So the WAV is staged under its final + stem and `reference.wav` — the file the façade gates on — is moved into + place **last**, after the artifacts and the manifest: a failed extractor + leaves a directory that is not a voice. `--force` retires an existing pack + whole to `.AGENTTRASH/` instead of touching bytes; nothing in this script + deletes. +- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B + per 12.5 Hz frame and `enc_dim 2048 × f32` = 8,192 B. That makes the + **effective crop window 19.84–25 s**, narrower than the WAV law's 18–25 s, + and a too-short crop is refused before the extractor runs. An engine bump + that changes either number fails the build by design. + +**voice stack audit: three faults that no gate could see** + +- The voice stack had never been run end to end, so the whole path was + audited as static code against the tools it calls, and three faults came + out of it. All three are the same species: a step that **reports success + while doing nothing**. +- **The build would not have happened.** `Containerfile.engine` created its + unprivileged user with `adduser`; the pinned ubuntu:24.04 base has no + `adduser` at all — proven in the base on Dankclaw, not argued from memory. + Replacement: `useradd --system --uid 10000 --home-dir /app --shell + /usr/sbin/nologin tts`; its `SYS_UID_MAX` warning is cosmetic and is the + price of matching the façade's uid. +- **The warm pass would not have warmed.** The unit hands its env file to + podman via `--env-file`, which performs no shell parsing and takes the + literal remainder of the line, while `source` strips one quoting layer. + `WARM_TEXT` therefore reached the container with its quotes, the JSON was + invalid, and every voice would log "registered, but the warm synthesis + failed" beside an `active` unit. Fixed in the hook; the bug survived the + first draft precisely because every existing test used `source`. +- **A rebuilt image would not have been rebuilt.** `build_compose_image` + skips `podman build` on a matching `cfhash` label, and that hash covered + the `Containerfile` alone — so editing `app.py` or `tts-engine-warm.sh` + (both `COPY`'d) skipped the build while the deploy reported success and + `/health` answered with old code. Each case arm now hashes the files its + `Containerfile` copies. +- Docs carry the lessons where an operator will hit them: a **Locally built + images** section in [[deployment]], two new troubleshooting lines in + [[Frank TTS Engine]] (all-voices-cold = quoted `WARM_TEXT`; façade 502 + beside 200 = model loaded ≠ voices registered), and `Voice Packs.md` + step 4 corrected — the warm hook re-registers **every** pack on every run, + because the engine replaces a name it already holds, so a restart is what + an engine bump needs and not what a rebuilt pack needs. + +**frank-tts-engine deployed: it builds, it serves, and the warm hook died on the kernel's argument cap** + +- Merged as **#14** (`747c665`) and deployed `--service frank-tts-engine`; + the image built clean — the first real proof of the `useradd` fix — and the + unit was `healthy` after 31 s. +- Then the deploy reported `0 voice(s) warm`: + `/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long`. The + hook handed the registration body to curl as ONE argument; the kernel caps + a single argument at 131,072 bytes (`MAX_ARG_STRLEN`) and a shipped ~955 KB + `reference.wav` base64's past that, so curl never executed and the hook + logged that as "HTTP no response". Fourth silent fault of the voice stack, + same species as the other three: `/health` stayed green throughout. Fix: + the body goes to a `mktemp` file and curl gets `@file`; the regression test + is the honest shape — real listener, real `curl`, a 300 KB fixture pack — + so it fails with the same kernel error the host produced. Every shipped + pack sits well over the ~96 KB cliff, so no small fixture could ever have + seen this. +- **A trial law did not survive deployment.** Against an empty registry the + first synthesis of a freshly registered voice cost ~1.9 s, the next ~1.2 s. + The trial's 6.2–8.0 s cold / 0.5 ms restore ladder, which the warmer's own + header repeated, is retired: warming buys registration — mandatory, because + the registry is memory-only — plus about **0.6 s**. Corrected in + [[Frank TTS Engine]] law 1 and its troubleshooting table, and in + [[Local TTS and Voice Cloning]] and [[Voice Packs]]. +- Sweep afterwards: the retired ladder still lived in six places the first + pass missed — both quadlet files, `scripts/frank-tts-engine.env`, + `scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row, the + research brief's consequences paragraph, and one test comment. All + re-keyed to the measured claim. The trial entry earlier in this log is + left as written — a dated worklog entry is a record — and this line is the + correction that points at it. + +**7772 is live and the harness speaks through it** + +- Merged **#15** (`0315a62`, the warmer's argument-cap fix) and **#16** + (`d2ce669`, the comment sweep), then deployed both voice services in order + (`--service frank-tts-engine`, then `--service frank-tts`); Playwright gate + 33/33 twice. Deployed-state proof from a LAN client: `/health` → + `omni-http` with all three voices, and the same sentence in `narrator1` vs + `narrator2` came back with different sha256 and different RMS — the name + actually selects a voice. +- **The harness `speech` role moved off Kokoro**: `config.yml` + `modelRoles.speech: frank-tts (dankclaw)/frank-tts`. No voice key moves + with it. +- Two harness laws learned by breaking it first (both now on [[Frank TTS]]): + an explicit speech role has **no fallback candidate**, so a dead endpoint + errors instead of degrading; and `models.yml` loads at process start while + `config.yml` is read live, so a session older than the `frank-tts` row + fails exactly like a dead endpoint. +- Corrected the same evening, by breaking it in a restarted session: a + harness call that names no voice reaches 7772 with **no `voice` key at + all** — `400 unknown voice None; available voices: …` — because no setting + feeds one. The endpoint therefore grew `DEFAULT_VOICE` (its own PR): a + voice-less request speaks the declared pack, an unknown name still 400s + with the list. `speech.voice: narrator1` — briefly recorded as wiring — + went back to `am_fenrir`: that key is the vocalizer's Kokoro enum, so a + pack name there breaks speak-aloud and does nothing for synthesis. +- Status sweep: six pages still said the endpoint did not exist — both vault + descriptions, the two `index.md` bullets, the Open Notebook blocker (the + voice leg works; open-notebook itself is what is missing), the engine + page's deploy order (now a deploy record), and the homepage card, which + lost `wip: down` so the admin panel probes 7772 for real. + +**the default voice shipped: a harness voiceover needs no voice** + +- Merged **#17** (`b590695`) and **#18** (`7889a68`) and deployed + `--service frank-tts --service homepage` in one dispatch, Playwright gate + 33/33. The call that had failed is the one that proves the fix: same + device, **no** `voice_id` — `Saved 430124 bytes (…, voice=default, + codec=wav)` — and the façade journal says what actually spoke, + `speech voice=narrator1 (default)`. The harness prints `voice=default` + because it never had a name to print; the voice that answered is the + operator's `DEFAULT_VOICE` line in `scripts/frank-tts.env`, and the journal + keeps them distinguishable forever after. +- The page gained a **Default voice** row in its contract table and + [[#Naming a voice]] now states that a call naming nothing is the harness's + normal shape, not a client bug — the misconception that produced the 400 in + the first place. + +**OpenWebUI "voice narrator1 not found": the rejector is in the browser** + +- Frank's toast — `Error: Voice "narrator1" not found. Should be one of: + af_heart, …, bm_fable` — looked like frank-tts refusing the name. It is + not: that 29-name enum is the voice pack of **Kokoro.js running in the + page** — OpenWebUI 0.11.4 ships that implementation in its own front end. +- Instruments that separated the two paths, in order: the admin config DB is + correct (`audio.tts.engine=openai`, base `http://host.containers.internal:7772/v1`, + `voice=narrator1`); from inside the container the exact OpenWebUI body + returns **200 `audio/mpeg`**; and `podman logs open-webui` holds **zero** + `audio/speech` requests for the failing click. A server-path failure leaves + a log line and a frank-tts error — a `browser-kokoro` failure leaves + neither — and no container env or `user.settings` row carries an override, + so the engine choice is per-browser storage, not fleet state. +- Fix is a UI click, not a deploy: Settings → Audio → TTS Engine → + **Default** hands synthesis back to frank-tts and the cloned voices; + choosing `af_heart` keeps in-browser Kokoro, which cannot clone (fixed + pack). Recorded on [[OpenWebUI]] under *Text-to-speech (frank-tts)*. +- Also in the window: **PR #22 merged** (`0c7fe8b`, `dank/command-center`) so + `agent/extensions/dankvoice.ts` landed on the auto-load path — proven in a + plain `omp` session with no flags: `/say merged build check` → WAV to the + player, temp dir cleaned. Sessions open before the merge need a restart to + pick the command up. + +**frank-zealot: 2 × 262,144 over unquantized f16 KV** + +- 2026-10-18 — frank-zealot: shape cut to 2 × 262,144 over unquantized f16 KV + (the 786,432 pool cannot fit exact KV on the 3090; concurrency 3 → 2, and + MTP is now doubly out — off by the 13:16 measurement, and exact KV bills a + draft-context f16 pool over the whole context). See [[frank-zealot]]. +- Arithmetic correction to the repo's long-standing figure: "the f16 pair + would need **26,112 MiB**" doubled the q8_0 number *including* its 1/32 + scale overhead — q8_0 is 17,408 B/token (1.0625 B/element), so f16/q8_0 is + **1.882×**, not 2×, and exact f16 over 786,432 is **24,576 MiB**. The + conclusion stands (three f16 windows cannot boot this card); the number was + wrong. +- New gate in `tests/frank_zealot_unit.bats`: dtype × pool plus the measured + non-KV terms must fit the driver-addressable card with **≥ 1 GiB** of slack + — and the same gate proves it refuses the impossible arm (f16 at 786,432), + not merely passes the shipped one. + +## 2026-10-17 +**frank-zealot: Bonsai-2-27B PQ2_0 trial + cutover (2 × 200,704, q8_0 KV)** + + +- 2026-10-17 — frank-zealot: **Ternary-Bonsai-2-27B (PQ2_0) cutover**, two + slots at 200,704 over q8_0 K + q8_0 V on the RTX 3090, fetched by + `scripts/download-frank-zealot-bonsai2.sh` (exact bytes + sha256, + `--verify-only`). See [[frank-zealot]]. +- Runtime door: the file is `file_type 141` with 402 tensors of custom GGML + type **142** plus `prism.hadamard.*` metadata — stock llama.cpp refuses + it, so the PrismML vendor build is **forced, not chosen**; the vendor + build serves q8_0 V under flash attention, so the source rebuild with + `GGML_CUDA_FA_ALL_QUANTS=ON` was cancelled as unnecessary. +- Trial-window law cost one boot: the scratch env invented + `KV_CACHE_TYPE` where the serve script reads + `CACHE_TYPE_K`/`CACHE_TYPE_V`, so the script's `:=f16` default silently + won and every arm booted f16. A silently-defaulted knob turns a whole + trial window into a different experiment. +- New fleet laws recorded (AGENTS.md + runbook): the mmproj's worst-case + VRAM is NOT inside the engine's fit projection; the first request after a + boot is a warmup artifact; a 64-token thinking budget makes a thinking + model look blind; the Bonsai-2 template reads `reasoning_effort` + (xhigh/medium/low, raises otherwise) so the harness effort fold is + load-bearing on this endpoint, and it reads `preserve_thinking`. + Recurrent DeltaNet state bills ~150 MiB **per slot** — pool arithmetic + alone misses it. +- Base alignment: **Frank chose the OFFICIAL censored payload** — publisher + provenance over refusal-freedom. The 9B it replaced was uncensored, so + the homepage card and its warning contract were re-keyed: zealot's card + now carries the base-alignment notice, the shared legal warning belongs + to dragoon alone. The CRACK re-export (byte-identical geometry per header + census) stays on disk as the one-line `MODEL_FILE` fallback if refusals + bite. +- Two live-path stragglers the cutover had to catch: + `serve-frank-zealot.sh` still DEFAULTED to the retired payload's paths + (a defaulted run would have served the old model — now re-keyed to the + shipped trio), and the OpenWebUI `model` row's `meta.description` is live + DB state (re-keyed with a DB backup; no `open-webui` restart forced — + forcing it logs every family session out for one cosmetic string). + +**dankserver disk purge: Swift w4dn8 rollback + orphan caches (docs re-key)** + + +- Ask: "dump the swift rollback and the GLM GGUF — make sure the model + caches and agenttrash are empty as well." Preflight first: `lsof` empty + on both payloads, the serving trio names only the live path, and a podman + layer census proved no deleted path backs an image container (the 121 GB + layer is the ENGINE image, not a model blob). Staged, then purged with + Frank's approval: the two retired payloads, derived kvcache tiers, stale + compile caches, and every `.AGENTTRASH` on the host. +- Recovery: the retired w4dn8 rollback payload is byte-exact on the Hub — + rollback is now a **re-download, not a local file**. Corpora (grams, + kld-ref, calibration JSONL) deliberately KEPT: they are the re-convert + and KLD-gate inputs. +- Post-checks: frank-archon active and `/health` ok, serving payload + byte-size intact, the live kvcache tier untouched, no process references + a deleted path. +- Docs re-keyed: AGENTS.md fleet row + frank-archon section + both runbook + index rows; the live runbook's Rollback rewritten (HF re-fetch procedure — + the fetch script's REPO/RAD_NAME/STAGE/DEST/SIZE/SHA256 constants are + hardcoded, so retargeting at the mirror is a script edit); the retired + runbook's header corrected (was still "LIVE", never updated at + retirement). + +**frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool** + + +- 2026-10-17 — frank-dragoon: cutover to the native v2 abliterated halogen + payload with the card's memory-saving KV pool; census first: the n-gram + table was ALREADY disk-backed and the only real lever was the KV pool. + See [[frank-dragoon]]. +- **Filename trap caught before the fetch:** this repo name carried two + different artifacts 32 bytes apart (the writer-transplant era's file vs + the native conversion). Byte-exact gate + LFS sha256 in + `scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the + peonist n-gram/vision/tokenizer trio — the release publishes only the + main file, and a hardlink is a regular file, so the no-symlink serve gate + passes without a 47.7 GiB copy. +- **Companion risk adjudicated by the engine, not by argument.** The card + does NOT establish the stock peonist companions as matched to a native + conversion and publishes no source-matched pair, so the A/B ran the + checkpoint tools on a stopped service under one knob set + (`scripts/census-frank-dragoon.sh`) plus a behaviour battery + (`scripts/battery-frank-dragoon.py`): two sound models disagreeing on + content, not a corrupt forward pass. The boot window ran through a + transient `FRANK_DRAGOON_ENV_FILE` drop-in — never a scratch port (one + iGPU, one server) — with the stop+mask law and a trap-guarded restore + (`scripts/trial-frank-dragoon.sh`). Masking fails on a relinked unit + file, as documented; stop alone was enough. +- Cost of the swap, stated plainly: two MAXIMUM-length conversations no + longer share the endpoint, the finetune's perplexity edge is gone, and + its reasoning came out compressed ~1.9×. Bought: RAM back as page cache + for the disk-backed n-gram table, faster cold prefill, clean retrieval, + and the base checkpoint itself instead of the finetune. The seam's pool + law relaxed from `pool == slots × ctx` to `ctx ≤ pool ≤ slots × ctx`. + +**frank-dragoon: ceiling and pool cut to 200,704; crawl reset fix** + + +- 2026-10-17 — frank-dragoon: context ceiling and KV pool cut to 200,704 + (49 × 4096, the per-slot rung frank-zealot runs; nothing in the fleet + needed the bigger window). See [[frank-dragoon]]. +- Harness moved WITH the ceiling (400-zone law): `models.yml` + `contextWindow` = ceiling − `maxTokens`; compaction fires at 85 % of the + window, and the 180k compaction target of the 262,144 era is unreachable + because the window itself is smaller. Seam re-keyed to prove the + invariant at the new pair — `ctx ≤ pool ≤ slots × ctx` and + `harness window ≤ ctx − maxTokens` — so a future raise of one without the + other fails the gate. +- Probes on the live endpoint: over the ceiling → `400 … tokens over` in + 0.86 s — preflight rejects before any prefill, so probing costs the + endpoint nothing. +- Deploy verdict: the endpoint dispatched healthy (`/health` 200), but the + `[3/3]` Playwright gate failed on `12-crawl` and the deployer correctly + said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was Plex + Web, signed out in the `qa-e2e` profile, running its own sign-in flow, + which broke the spec two different ways: (1) it redirects the top frame, + so Chromium cancelled the crawl's in-loop `page.goto("/")` with + `net::ERR_ABORTED` — and that reset sat outside the `try`, so one host + aborted the whole crawl; fixed with `backHome()`, which re-issues on that + one error class. (2) The same hand-off leaves the browser on + `app.plex.tv` (Plex's hosted surface for a session-less client), so the + `hostname == target` law could never settle; fixed with an explicit + `HANDOFFS` landing list. Plex itself was healthy throughout, and the + guard contract is still asserted for every vhost. Both symptoms recorded + as gotcha 6; the gate after both fixes went 33 passed with the crawl + green on the first attempt. + +## 2026-10-15 +**frank-archon: publisher iq4r payload swap (MODEL_FILE-only)** + + +- 2026-10-15 — frank-archon: publisher iq4r payload swap, `MODEL_FILE`-only + — `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`, the engine author's + published rad-convert of the OFFICIAL Qwen3.8-Flash-Next, **no other + argument changes**, mindful the operator session runs on the endpoint + being swapped. See [[frank-archon]]. +- Disk: Frank authorized one `rm` — the spent BF16 source and the + two-era-old fp8-native `.rad` left `/home/dank/models`; fp8-native is now + rebuild-only (fetch + build scripts); rollback for THIS swap is the + previous `.rad`, on disk, by reverting the commit. +- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes + + Hub LFS sha256, `--verify-only` default) as a transient unit. The cutover + **self-severed by design** — a detached `deploy.sh` survived the session + death when frank-archon restarted; engine healthy in ~100 s, and the + prefix tier RECOVERED across the swap (arch+tokenizer unchanged). +- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw + rebooted 01:41 and the gluetun boot-race dependency-failed `searxng`, + `marreta`, and `marreta-browser` (jobs consumed while gluetun's start job + retried; `Restart=always` never armed because the services never ran). + Cascade restarted; skill codified + (`gluetun-dependent-boot-race-recovery`). frank-archon's own health gate + passed every deploy. + +**authentik: two more human accounts (Dankclaw)** + + +- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, + no repo objects**, each `@franksdank.cloud` in the human-user + template shape (password via `set_password` on stdin only — never argv, + so nothing lands in `ps`), verified through the exact PasswordStage code + path; zero `proxysession` rows for either, so no claim purge was due. +- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user census + extended with pk 22–23. + ## 2026-10-13 - **frank-archon outage: a kernel knob, not the model.** The 2026-10-12 23:17 power failure on dankserver reset `/sys/module/ttm/parameters/pages_limit` @@ -117,6 +887,28 @@ tokens, per-stream + aggregate) so no future shape change can be priced on prefill alone. +## 2026-10-11 +**frank-archon: unquantized (bf16) KV at 2 × 200,000** + + +- 2026-10-11 — frank-archon: **unquantized (bf16) KV at 2 × 200,000** on + exact attention state, with pool, slab and host pool rebalanced around + the bigger cache. See [[frank-archon]]. +- Engine door: `--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the + whole cache, so llama.cpp's exact-K / quantized-V pair (what frank-zealot + runs) has **no radiance equivalent**. +- `--max-num-seqs 3 → 2` booted clean: the fleet's "seqs 2 cannot boot" law + is the retired tcclaviger vLLM fork's torch.compile `query_start_loc` + constraint, not radiance's — radiance booted seqs 2 in its fp8-native era + and the sweep had 2 and 3 tied inside noise. +- Harness law re-learned: radiance puts the thinking block in + `reasoning_content`, so a probe that reads `content` on a small budget + sees empty text with `finish_reason: stop` — a healthy server scored 0/2 + until the probe set `chat_template_kwargs.enable_thinking=false`. Tool + committed as `scripts/probe-frank-archon-kv.py`. +- The fp8-vs-bf16 control arms and the open keep-bf16-or-take-cheap-fp8 + call are recorded on [[frank-archon]]; the 2026-10-13 section below (in + file order above) closes that call in favour of bf16. ## 2026-10-10 - **Fleet service rename: the StarCraft names became the SERVICE names.** @@ -504,6 +1296,243 @@ positive, KV pool >= 400,700 tokens at the measured 7,892 B/token). Runbook: `serverAppConfigs-docs/LLM/frank-archon.md`. +## 2026-10-02 +**frank-dragoon cutover: vLLM MiMo -> ROCmFPX Q8_0_ROCMFPX_AGENT + MTP** + + +- 2026-10-02 — frank-dragoon: vLLM MiMo → ROCmFPX `Q8_0_ROCMFPX_AGENT` + payload + MTP from the repo's own bundled head, one session at the native + 262,144. See [[frank-dragoon]]. +- The runtime is **forced by the file**: tensor type 103 + (`GGML_TYPE_Q8_0_ROCMFPX`, ftype 115) is unreadable by stock llama.cpp, + gufo and every other loader on the fleet — only the kingjones30/ROCmFPX + fork can read it, so the engine is pinned to commit + `36e9acd40e10a87cd3c3ef8ec734668757dc8520` and built by + `scripts/build-rocmfpx.sh`. The AGENT routing variant was chosen over the + card's plain Q8_0 because it keeps 187 output-head tensors at plain Q8_0 + instead of 2, which measured better MTP acceptance. MTP was the whole + performance story of the shape (roughly +120 % decode, ladder on the + runbook). +- Rollback: git tag `frank-dragoon-gufo-q4kxl-20261001`; the retired + vLLM-era fetchers moved to `old/`. Runbook [[frank-dragoon]]. + +**frank-zealot: Q5_K_M, then three contexts + vision, sized by two boots** + + +- 2026-10-02 — frank-zealot: Q5_K_M payload, then three contexts + vision + in one day, sized by two boots (`CTX_SIZE=540672 PARALLEL=3`, f16 K + + q8_0 V under flash attention, `--mmproj`). See [[frank-zealot]]. +- **The pool came from two boots, not a formula.** Differencing the boots + prices the non-KV floor — and prices the vision tower at **~1.4 GiB for + a 918 MB file**: the encoder's own buffers ride with it. The first 3-slot + deploy booted healthy but left less than the ordered 2–3 GiB band free, + which is why the seam now prices non-KV at the measured value and + REJECTS that shape. +- Three contexts cost window, and that is the tradeoff Frank was told: + 180,224 tokens per conversation, not the native 262,144; a longer prompt + is a hard `400 exceed_context_size_error`, so `models.yml` follows the + slot, not the card. +- **Vision contract gotcha:** `/v1/chat/completions` ignores the legacy + top-level `images` field — that request answered "Brown." for the same + red image, i.e. it generated from text alone. Only the `image_url` + content part reaches the model. `/completion` accepts `images` but this + build still requires `messages`. +- Deployer gates on the projector too (`LLM_SMALL_MMPROJ`), so a missing + mmproj cannot silently ship a blind endpoint; `MMPROJ_FILE=` + (explicitly empty) in the env file is the documented text-only opt-out + and returns the tower's ~1.4 GiB to KV. +- Reclaim staged, **not purged, awaiting approval**: + `~/.AGENTTRASH/20261002-retired-llm-payloads/` of superseded payloads. + Runbook [[frank-zealot]]. + +**frank-archon shape D: 3 x 200704, fp8, pool sized for 2 contexts** + + +Frank's order: shrink frank-archon to a 200704-token window with about two +resident full-length contexts, cut the KV pool, and drop MNBT to 1024. The +first attempt (`30904ba`, 2 x 200704 + SEQS=2) **crash-looped**; the revert +went live as `e460bbb`, and the shape was then re-applied one law at a time on +top of the proven-good base, root cause first. + +- **ROOT CAUSE: `--max-num-seqs 2` cannot compile this model. It was never + OOM, never MNBT, never the KV dtype.** Four single-variable boot probes + against the then-deployed shape (production stopped for each, candidate + booted AS the service): 262144 x SEQS 2 fails at BOTH MNBT rungs with + `torch.fx.experimental.symbolic_shapes.ConstraintViolationError: + Constraints violated (L['query_start_loc'].size()[0]) ... specialized it + to be a constant (3)`, raised from AOT `build_guards(strict_error=True)` + during `determine_available_memory -> profile_run -> _dummy_run`, ~7 min + in; SEQS 3 is healthy at both rungs. `model_runner.py` sizes that input + buffer `max_num_reqs + 1` and `_dummy_run` sets `num_reqs = + min(max_num_batched_tokens, max_num_reqs)`, so its width is **SEQS + 1** + for ANY `MNBT >= SEQS` — 3 at SEQS 2, exactly the constant in the + message; MNBT only moves the compile RANGE, which is why the identical + error appears at both rungs. `--enforce-eager` bypasses the path but + drops CUDA graphs. The 2026-10-01 bf16 2 x 262144 failure was this same + error class; its message named "KV dtype and SEQS" together and nobody + bisected, so the dtype was blamed — it was a red herring. +- **THE ARENA LEVER IS `max_model_len`, NOT `--kv-cache-memory-bytes`.** + The planner sizes expert cache as `budget - non-expert - KV reserve - + graphs - overhead`, and the KV reserve is registry-priced at **9190 + B/token x max_model_len**, independent of the pool override: d(reserve) + equalled d(arena) exactly, and the multi-GB pool cut bought **zero** + arena slots. The previous session's premise — "the pool cut becomes + arena, ~7,500 slots" — was wrong; the pool cut is a runtime-HEADROOM + lever (the shape it replaced died with `0 bytes free` on a real + 995-token request). +- **Final shape (option D):** the argv keeps the only slot count that + compiles; the POOL caps resident conversations (ledger on + [[frank-archon]]). Measured live, one prediction corrected: the arena + prediction landed exactly, the concurrency one did not — **the per-token + KV price is shape-dependent** (7,701 B/token at MAXLEN 262144 vs **7,892** + at 200704, ~2.5 % dearer). Never extrapolate a concurrency figure across + geometries. Smoke probes confirmed the 3 slots real, native `tool_calls`, + and that `reasoning_effort` accepts only {xhigh, medium, low}, answering + 400 for anything else. +- **Harness contract followed the window:** `models.yml` `contextWindow` + 262144 -> **200704**; side effect: frank-archon's compaction request fits + inside a frank-zealot slot again. A comment-only follow-up `295d578` was + deployed so the per-service revision marker sits at HEAD. Rollback for + the pre-shape-D fp8+MTP4 era stays git tag + `frank-archon-fp8-mtp-20261001`. Runbook [[frank-archon]]. + +**plex: the readiness gate probed HTTP against a TLS-only service** + + +Found by accident, and it had been latent since the secure-connections posture +was set: `deploy_plex` gated on `curl -fsS http://127.0.0.1:32400/identity`, +which **cannot** succeed. PMS drops plain HTTP off-loopback - the probe's +connection arrives on the container's eth0 - so Plex accepts the TCP connection +and then answers with nothing, and curl reports "Empty reply from server" (52). + +- Trigger: a bare `./scripts/deploy.sh dankserver` (no `--service`) drains + the whole pending backlog, not just the service you were thinking about. + plex's per-service revision had been behind long enough that this gate + had never run against the current posture; it burned its full 600 s and + failed the run, and every service after plex in the order was left + undeployed. +- **Plex was healthy the entire time.** `https://127.0.0.1:32400/identity` + returned 200 with the MediaContainer XML in 44 ms while the gate was + still timing out; the in-container plain-HTTP path also answers 200 — + loopback is the one path PMS serves in the clear, which is exactly why + the host-side probe could not see the difference. +- **The repo already knew this law, in a different file:** + `tests/caddyfile.bats` pins the plex vhosts to a TLS upstream with the + comment "PMS drops plain HTTP off-loopback". The deployer's gate never + got the same treatment. Fixed to `curl -fsSk --max-time 3 + https://127.0.0.1:32400/identity` (`-k`: PMS serves its own self-signed + cert on 32400). Re-deployed `39f0e0d`: plex recorded deployed, e2e + 33/33, frank-archon untouched (`health=200`, NRestarts=0). +- **Operating lesson: `deploy.sh ` with no `--service` is a + fleet-wide drain.** It restarts every service whose recorded revision is + behind HEAD and whose paths changed since. To touch one service, name it. + +**frank-archon shape E: `--expert-precise-memory` (+1,026 arena slots), a mounted compile cache, and the P2P answer** + + +Shape D left one question open — the plan's overhead term was reserving 2.35 GiB +of compile headroom the boot never used — and the answer to Frank's free-VRAM +question arrived together with a boot-cost bug that had been costing 181 s on +every restart since the cutover. + +- **`--expert-precise-memory` is real and it fits (option A).** Boot-probed on + the shape-D base with production stopped, the candidate booted AS the service + (`FRANK_ARCHON_ENV_FILE=/tmp/probe-b.env bash scripts/serve-frank-archon.sh`, + trap-guaranteed restore), the numbers read off the engine's own plan lines: + + | plan line | shape D | shape E | + |---|---|---| + | `runtime overhead (non-torch + buffers + activation)` | 7.60 GiB | **5.25 GiB** (`precise (gfx1201 TP2 nbt4096/seqs16/len262144)`) | + | `expert cache space` | 14.87 GiB | **17.22 GiB** | + | slots | 6,496 | **7,522** (+1,026, +16 %) | + | `swap region` | — | 13.7 GiB / 5,986 slots, 133-134 per layer | + | boot | — | **healthy in 420 s**, no OOM | + + The flag also prints `'expert_precise_memory': True` in `non-default args`, so + a silent argparse drop cannot hide. It was refused on 2026-10-01 only because + the 2 x 262144 bf16 shape was already 0.13 GiB under the ceiling — there the + flag was the last stack over the line, not the cause. +- **The pool was NOT cut (option A) because the pool is not an arena lever.** + A 2.2e9 arm was probed in the same window and reports `277,686 tokens, + 1.38x` — measured, rejected as the shipped value, kept as the conservative + fallback. With 3.15e9/rank the endpoint keeps shape D's 1.99x concurrency and + gains the arena, which is the shape D follow-up had actually been looking for. +- **THE COMPILE CACHE WAS BEING THROWN AWAY ON EVERY RESTART — 181 s each.** + Measured: a fresh container has **no `/root/.cache` at all** (`No such file or + directory`); after our boot it holds `/root/.cache/vllm` at + **1,154,714,333 B in 4,665 files** plus `/root/.cache/comgr` at 229 MB, inside + the ephemeral layer, and `podman rm -f` (every deploy, restart, reboot) + discards them. The engine names the loss itself: `Compiling a graph for compile + range (1, 1024) takes 173.19 s` ... `torch.compile took 181.20 s in total` ... + `init engine (profile, create kv cache, warmup model) took 217.26 s + (compilation: 181.20 s)`. The serve script now mounts + `FRANK_ARCHON_CACHE_DIR` at `/root/.cache` — the **parent**, so the vLLM + AOT/torch_compile tree (`VLLM_CACHE_ROOT`, default `/root/.cache/vllm`) and + the comgr JIT cache both persist, and nothing is hidden because the image has + no such directory. Same class of fix as the recipe's + `/cache/{triton,vllm,inductor}` volumes; the compile RANGE is part of the + cache key, so an MNBT change costs one recompilation, not a permanent tax. +- **`GPU_MAX_HW_QUEUES=2`** (HSA hardware queues per GPU) added at Frank's + direction. Verified a real knob rather than a no-op — the literal is read from + `/opt/rocm/core-10.0/lib/libhsa-runtime64.so` — and shipped deliberately + alongside the 32x64 gather geometry. **Not A/B measured.** +- **P2P: the hardware supports it, and the engine is already using it.** The + live log shows `r4d all-reduce INSTALLED (rank=0 max=49152KB finegrained=T…)` + on BOTH ranks and `Using ['R4D_AR', 'PYNCCL'] all-reduce backends` — R4D's + P2P path, with RCCL as fallback, measured at **45.274 GB/s bidirectional**. + Root ports `00:01.1`/`00:01.3` both report ACS `ReqRedir+`, so P2P hairpins at + the root complex; there is no XGMI. The old FAIL predates the 2026-09-20 board + swap. Kernel support confirmed: `CONFIG_HSA_AMD_P2P=y`, `CONFIG_PCI_P2PDMA=y`, + `CONFIG_HSA_AMD=y`, `CONFIG_ZONE_DEVICE=y`, `iommu=pt` on the cmdline. +- **Recipe diff (Frank's 3 x R9700 compose vs this podman run), ranked:** + - *Acted on:* persistent compile caches (above — the one real operational + gap), and `GPU_MAX_HW_QUEUES`. + - *Retracted:* pinned `--compilation-config cudagraph_capture_sizes` was on my + list; measured, it is **not** a gap here — capture is 3 PIECEWISE + 2 FULL + graphs, `Graph capturing finished in 3 secs, took 0.31 GiB` (0.11 GiB on the + lighter rank). No 15-52-size ladder to cut. + - *Not acted on — measured candidates, none shipped (no bench window; these + are CPU-side and our decode is gather-bound, so a spin-wait stealing cores + from the gather threads is plausible but unproven):* `OMP_NUM_THREADS=8` + (defaults to all 16 logical CPUs on this 7700X), `HSA_ENABLE_INTERRUPT=1`, + `HSA_ENABLE_MWAITX=1`, `HSA_ENABLE_SDMA`, `HSA_NO_SCRATCH_RECLAIM`, + `HSA_ENABLE_IPC_MODE_LEGACY`. All six HSA names verified real — the literals + are read out of `/opt/rocm/core-10.0/lib/libhsa-runtime64.so` — so they are + knobs, not no-ops; any of them needs its own A/B before shipping. + - *Deliberate divergence, keep ours:* no MTP (the recipe has it; measured + fastest here), `qwen3_xml` vs `qwen3_coder` (ours verified live), no + `--override-generation-config` (server-side sampling defaults), published + port + Caddy rather than `network_mode: host`. + - *Non-gaps:* `shm_size 32gb` (ours is `--ipc=host`), `memlock -1` (same), + `SYS_PTRACE` (same), `/dev/kfd` + `/dev/dri` (same), `--enable-chunked-prefill` + (V1 default), `plecache` volume (the flag is not even passed), and + `ROCR_VISIBLE_DEVICES=1,2,3` vs our `HIP_VISIBLE_DEVICES=0,1` — verified + equivalent here: `rocm-smi --showid` puts the two R9700s at GPU[0]/GPU[1] and + the 7700X iGPU at GPU[2], so our indices are the right two cards. +- **Harness note, because it cost two probe attempts:** a probe that reported an + instantaneous candidate death was a logging artifact of the inline SSH script, + not a crash — the container and its serve script must be polled by recorded id + and their logs dumped on the exit path, or a failure looks like a bad flag. + The flag was fine both times. +- Live: `health=200`, `NRestarts=0`, arena 17.22 GiB / 7,522 slots, KV 398,658 + tokens / 1.99x. Docs: [[frank-archon]]. +- **Deployed `1ac4d48`, e2e 33/33, and the live plan matched the probe exactly:** + `'expert_precise_memory': True`, `runtime overhead … 5.25 GiB`, `expert cache + space 17.22 GiB / 7522 slots`, `swap region 13.7 GiB / 5986 slots`, `GPU KV + cache size: 398,658 tokens … 1.99x`, pool 3,150,000,000, MNBT 1024. The + precise overhead and the arena growth are now production numbers, not a + probe's projection. +- **The compile-cache fix was then proven on the same endpoint, two consecutive + boots of the deployed shape:** cold `Compiling a graph for compile range + (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s`; warm `Directly load + the compiled graph(s) for compile range (1, 1024) from the cache, took + 2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile, + create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)`. The host + dir held 1.4 GB in 4,707 files after the cold boot. **Time to `/health` is + dominated by the 172.8 GiB payload load** (the warm boot still needed 490 s), + so the honest claim is ~184 s of inductor/AOT build recovered per restart — + not a proportional boot cut. + ## 2026-10-01 - **frank-archon rolled BACK to Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4; GLM/DFlash2 declared failed and purged.** The DFlash2 drafter booted and @@ -523,6 +1552,276 @@ KV-cache patches 02/03/04/04b survive in `scripts/patches/` (generic unifier/tailpin/grouping/registry fixes, apply-verified). +**frank-archon RESTORED to fp8 3 x 262144 (bf16 boot fails)** + + +bf16 (--kv-cache-dtype auto) at 2 x 262144 did not stay up, with OR without +--expert-precise-memory: + +- with precise: NRestarts 1->3, each attempt ~6.7 min then + `Deactivated successfully` (API server drains after an EngineCore fatal + error) — cause not captured, the serve script rm'd the container. +- without precise (default 7.60 GiB overhead, arena 14.3 GiB): NRestarts + climbing, container `Exited (1)`, and the journal finally showed the real + error: + `RuntimeError: Worker failed with error 'Constraints violated + (L['query_start_loc'].size()[0])! You marked L['query_start_loc'].size()[0] + as dynamic but your code specialized it to be a constant (3)'` + — a torch.compile/dynamo dynamic-shape constraint violation, NOT an OOM. + `~/.cache/vllm` holds only `modelinfos` (16K) and each `podman run` gets a + fresh container filesystem, so a stale compile cache is ruled out: the + conflict is produced by this configuration's own compile path. + +Differences from the last shape measured serving (fp8 3 x 262144, MNBT 2048, +KV 6.1e9 -> 792,069 tokens / 3.02x / arena 6,248 / 3.59 GiB free): KV dtype +(fp8 -> bf16) and SEQS (3 -> 2). Both are candidates for the dynamo trigger; +`--enforce-eager` is the bypass that would sidestep the constraint path. + +Restored: fp8 3 x 262144, MNBT 2048, KV_BYTES 6,100,000,000, no precise flag. +Next: bisect bf16 vs SEQS in a window (bf16 + SEQS=3 isolates the dtype). + +**frank-archon: --expert-precise-memory + bf16 = boot crash loop** + + +**Reverted.** The two changes Frank asked for were applied together and the +result crash-looped: `NRestarts` climbed 1 -> 2 -> 3, each attempt running +~6.7 minutes then logging `Deactivated successfully` (vLLM's API server drains +and exits 0 after an EngineCore fatal error, so `podman wait` returns 0; the +new `Restart=always` turned a silent death into a visible loop). + +The budget arithmetic explains it without needing the error text, because the +arena is DEVICE-resident (the fp8 3-window shape's plan predicts 28.27 GiB and +that is exactly what `rocm-smi` measured): + +| line | bf16 2 x 262144 + precise | fp8 3 x 262144 | +|---|---|---| +| non-expert weights | 6.28 | 6.28 | +| pinned pool | 6.80 | 5.68 | +| CUDA graphs | 0.55 | 0.55 | +| arena | 16.65 (precise: overhead 5.25) | 14.31 (default: overhead 7.60) | +| non-torch | ~1.45 | ~1.45 | +| **total** | **31.73 of 31.86 GiB** | 28.27 (measured) | +| free at boot | **~0.13 GiB** | 3.59 | + +`--expert-precise-memory` hands the 2.35 GiB compile/graph-capture headroom to +the arena, and bf16 simultaneously adds 1.08 GiB of pinned pool; together they +put graph capture ~0.13 GiB from the ceiling, where every transient allocation +dies. So the flag is dropped and bf16 2 x 262144 ships with the default +overhead: arena 14.31 GiB, ~2.4 GiB free at boot. + +**Lesson:** the registry's "compile headroom" is not slack to be reclaimed by +default — it is the budget for the phase that allocates last. Reclaim it only +by paying for it elsewhere (smaller pool, fewer sessions, or a shorter context). + +**frank-archon: 2 bf16 windows + --expert-precise-memory** + + +Frank's final shape for the night. Two changes at once, both measured in +principle before the boot: + +- **KV is bf16 (`--kv-cache-dtype auto`), two full-size windows (2 × 262144).** + bf16 measures **13,839 B/token** (from the bf16 3×241664 boot: + 11,047,010,304 B -> 798,280 tokens), decomposing as **12,276 attention + + 1,563 mamba/side caches** against fp8's **6,138 + 1,563 = 7,701**. So + 524,288 tokens needs 7.256e9 B; pinned at 7,300,000,000 (~527,500 tokens, + 3,206 margin). Cost: the pool goes 5.68 -> 6.76 GiB per rank. +- **`--expert-precise-memory`.** The registry's default overhead for this + architecture is 7.60 GiB = the fitted 5.25 GiB (gfx1201 TP2 + nbt4096/seqs16/len262144) + 2.35 GiB of compile/graph-capture headroom. + The flag selects the fitted figure, which moves only the plan's `room` term + — arena 6,248 -> ~7,278 slots (~1,025 more, at 2.344 MiB/slot) — at the + cost of boot-time capture slack. The overhead formula adds 0.22 GiB per + 1024 nbt above 4096, so MNBT 2048 pays nothing. + +Budget note worth keeping: the arena is `(budget - non_expert_weights - +kv_reserve - graphs - overhead) // expert_bytes`, where `kv_reserve` is the +planner's OWN one-max-model-len reserve (2.48 GiB at TP2) and does NOT scale +with SEQS. The pinned `--kv-cache-memory-bytes` therefore does not shrink the +planned arena; it comes out of free VRAM instead. + +**frank-dragoon: Q4_K_XL payload + 4 x 163840 sessions** + + +Frank's call. Payload swapped from ajgazin's **abliterated** Swift-1.5 +UD-Q8_K_XL (29.3 GiB) to **unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL** (16.35 GiB, +sha256 3f227079…b01e, vanilla/Apache-2.0 — the endpoint is no longer +uncensored). New fetcher `scripts/download-frank-dragoon-q4.sh`; the DFlash2 +drafter and its fetcher are unchanged. + +Shape: **`--sessions 4 --context 163840`** with unquantized KV. gufo has no +KV-quantization lever at all (`execution_policy.hpp` pins `kv_cache_storage` +to `kFp16`), so "unquantized" is inherent. + +KV arithmetic (why this fits where one 262144 session used to sit): +f16 KV costs **64 KiB/token** — 16 full-attention layers (the hybrid is +`full_attention_interval: 4` over 64 layers) x 4 KV heads x 256 head_dim x +2 (K,V) x 2 bytes; the other 48 layers are linear-attention with a +per-SEQUENCE fp32 state (~144 MiB), not a per-token cache. 4 x 163840 = +655,360 tokens = 40.0 GiB of KV, and each session is exactly 10 GiB. Budget: +weights 16.35 + KV 40.0 + drafter 1.07 + linear state 0.56 + runtime ~4 = +~62 GiB against the 114 GiB GTT. + +**e2e gate flakiness (12-crawl) — do not panic at the banner** + + +`specs/12-crawl.spec.ts:23` ("every user open link reaches its app") failed the +deploy gate twice and then passed cleanly on a manual re-run: +- run 1 (deploy): `Expected "plex.franksdank.cloud"` / `Received "app.plex.tv"` + — Plex redirects its web UI to the hosted app; the same failure appeared on + the previous deploy too. +- run 2 (manual): `page.goto: net::ERR_ABORTED at https://www.franksdank.cloud/` + — the homepage answered `http=200` in 40 ms from the dev box at the same + moment, so this is a browser-level flake. +- run 3 (manual): **33 passed**, exit 0. + +So a red gate on this spec alone is NOT proof the fleet is broken, and the +deployer's `DEPLOY INCOMPLETE` banner can be raised by a flake. Re-run +`tests/e2e/run.sh` before treating it as real; the Plex redirect is the one +stable underlying issue and it lives in Plex's web-client/image config, not in +any repo service. + +**frank-archon bf16 no-spec cutover** + + +- `/scripts/frank-archon.env` — bf16 no-spec cutover, FINAL shape after the + night ladder: MAXLEN **241664** x SEQS 3, KV_BYTES **10,184,000,000**/rank, + KV_DTYPE auto (bf16), MNBT 1024, GATHER CHUNKS=32/LANES=64, UTIL 0.98; + FRANK_ARCHON_SPEC deleted (no spec). Engine measured at the winner: + "KV cache size: 732,359 tokens, Maximum concurrency for 241,664 tokens per + request: 3.03x". +- **The night ladder (Frank's method).** 262144 OOMed at CUDA graph capture + (`Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB + allocated by PyTorch) and 245760 OOMed identically; **237568 PASSED** + (KV 718,606 tokens) and the +4096 bonus step **241664 PASSED and is + PRODUCTION**. 241,664 is max possible on this box at util 0.98 — the + model's native 262,144 does not fit with a 3-seq contract. Every rung was + booted under a transient systemd unit at uid dank with + `LimitMEMLOCK=infinity` (the rootless bench probe masked the production + memory environment) with full engine-log capture. +- **Util stays 0.98.** The earlier 0.90 edit was wrong — lowering + `--gpu-memory-utilization` shrinks the engine's own budget rather than + buying slack; Frank forbade it and the winning ladder ran at 0.98. +- `/scripts/serve-frank-archon.sh` — `--speculative-config` line removed; + `--max-num-batched-tokens` and the `-e R4D_LRU_GATHER_CHUNKS/LANES` + container-env passthrough added; shape header rewritten. +- `/tests/frank_archon_unit.bats` — seam re-pinned: bf16-KV-present, + spec-absent-by-measurement, gather-32x64-present, MNBT-1024 tests; the + qualified-shape and flags tests flipped to the new env keys. +- Overnight redo campaign (prefetch OFF everywhere) tabled in + `/serverAppConfigs-docs/LLM/frank-archon.md` + with the 2026-10-01 cutover section: gather 32x64 +18.7%, MNBT flat + 1024-4096, KV pool law (7,926 fp8 / 14,047 bf16 B/token), arena flat zone, + six-slot refusal, bf16 3x262144 boot failure root cause (CUDA OOM at + KV_BYTES 11.3e9; boots clean at 11.047e9 — captured from journald `-o json`). +- `/AGENTS.md` — fleet table frank-archon row rewritten to the bf16 shape. +- `/serverAppConfigs-docs/LLM/index.md` — fleet intro + mermaid node + updated out of the GLM era. +- **Client contract fixed (thinking + tools).** The live endpoint returned a + null `reasoning` field and dumped the thinking text INLINE in `content` + with a dangling `` (measured: `'We need answer…Brief.\n\n\n391'`) + — the malformed thinking block clients choke on — because the deployed argv + had no `--reasoning-parser`. The GLM rollback (`d91379d`) had dropped all + three contract flags and the qwen rollback inherited the gap. Restored from + `ecaad36`'s values, confirmed against this fork's registries: + `--reasoning-parser qwen3`, `--enable-auto-tool-choice + --tool-call-parser qwen3_xml`, `--default-chat-template-kwargs + '{"enable_thinking": true, "preserve_thinking": true}'`. The seam's + "NO tool dialect" test is flipped to a PRESENT test. +- **Final shape (Frank's call): 5 seqs x the full native 262144 window at + fp8 KV.** 6 x 241664 at fp8 died (`Tried to allocate 10.70 GiB`, 9.18 GiB + free) — six full-length contexts do not fit at any KV dtype — so the + contract moved to five FULL windows: 5 x 262144 fp8 PASSED on the first + rung, KV_BYTES 10,388,766,720/rank, measured **1,350,182 tokens / + 5.15x concurrency** (39,462 tokens of margin). Seam re-pinned; the bf16 + three-context shape (241,664, KV 732,359 / 3.03x, measured earlier the + same night) is the documented max at 3 contexts. +- Harness `/home/blank/.omp/agent/models.yml` — contextWindow back to 262144 + with the five-window fp8 shape (compaction gap 183,501 + 36,864 = 220,365 + <= 262,144) and + `preserve_thinking` moved from a bare top-level body field to + `chat_template_kwargs` (the form vLLM actually reads). +- **Runtime-survivability correction: five slots -> FOUR.** 5 x 262144 fp8 + passed every boot gate (KV 1,350,182 tokens / 5.15x) but left 0 bytes free + at runtime; a real client request (995-token prompt, `max_tokens=261149`, + sampling params straight from models.yml) killed Worker_TP1 with + `torch.OutOfMemoryError: Tried to allocate 50.00 MiB ... 0 bytes free` and + the engine shut down. Boot is not proof of runtime survivability — after a + knife-edge shape change, send a large-`max_tokens` request before calling it + done. FOUR slots x 262144 fp8 (KV_BYTES 8,311,013,376/rank) returns ~1.93 + GiB per rank for activations/graph capture. Frank's call 2026-10-01. +- `/scripts/frank-archon.service` — **Restart=always, not on-failure** (plus + `StartLimitIntervalSec=0` and `OOMScoreAdjust=-500`). A runtime worker fault + makes vLLM's API server exit CLEAN, so ExecStart returned 0, systemd logged + "Deactivated successfully", and on-failure NEVER fired: the endpoint stayed + down until a human noticed. An explicit `systemctl stop` still wins, so + bench windows and deploys are unaffected. +- **Final shape (Frank's call, 2026-10-01): 3 x 262144, pool at the FLOOR, + MNBT 2048, rest to the arena.** MAXLEN 262144 x SEQS 3 = 786,432 tokens of + contract; measured fp8 costs 7,698 B/token, so 6.054e9 B is the pool floor + and the deployed 6,100,000,000 B leaves a ~5.9k-token margin while handing + the remaining ~2.2 GiB (versus the 4-slot shape) to the expert arena — the + decode-critical cache the fork sizes from the remainder. MNBT 2048 costs the + arena nothing (the sweep measured 6,499 slots at every rung <= 4096). +- **DCP investigated and closed** (see the parallelism survey in the runbook): + `--decode-context-parallel-size 2` is rejected at boot — DCP for GQA requires + TP > the model's KV heads (2), i.e. TP >= 4, i.e. four GPUs. Not a tuning + knob on this box. +- Production bring-up: the shape boots and serves on :7770 — engine log + verified (KV cache size, concurrency, MNBT 1024, gather 32x64) plus live + contract probes (reasoning split, tool_calls). Rollback target for the + 2026-09-29 fp8+MTP4 shape: git tag `frank-archon-fp8-mtp-20261001`. + +**frank-dragoon cutover: gufo -> vLLM TheRock, MiMo-V2.6 BF16 + cross-model MTP** + + +- **Question that started it:** can a Qwen3.5 MTP drafter attach to + frank-zealot, and how does vLLM run the FULL BF16 MiMo checkpoint? Answer 1: + not on frank-zealot — SGLang reads the MTP head from the TARGET checkpoint + and MiMo ships ZERO `mtp.*` tensors (base and INT8 alike). Answer 2: the + digest-pinned `kyuz0/vllm-therock-gfx1151` image (vLLM 0.30 / torch 2.11 / + TheRock ROCm 10) boots BF16 MiMo on the gfx1151 iGPU in ~200-240 s, and + vLLM reads the draft from a SEPARATE path — so cross-model MTP from + `Qwen/Qwen3.5-9B` (the distill's source, identical 4096/248320/32-layer + geometry) works. Frank's call: ship that as frank-dragoon. +- **Bench window (frank-dragoon stopped, one session, temp 0, 28,647-token + prefill / 512 decode):** base 17,876 t/s prefill / 13.1 t/s decode (13.1 + is ~94% of the LPDDR5x BF16 roof - decode here is bandwidth, not a bug); + +DFlash-8 7,976 / 17.4 (acceptance 59%/26% per position); +MTP-2 7,380 / + 19.8 (71%/44%, token rate 57.4%). Deployed MTP-2: +51% decode. Both spec + arms pay ~55-59% of prefill. `num_speculative_tokens` beyond 2 unmeasured. +- **Measured numerics law:** spec arms do NOT reproduce the no-spec greedy + text (bf16 batched verification reorders reductions; near-tie argmaxes + cascade). vLLM spec decode is distribution-preserving, not + bitwise-greedy-preserving. +- **Client-contract delta wired:** vLLM emits `reasoning`, not + `reasoning_content` - `models.yml` frank-dragoon block now sets + `reasoningContentField: "reasoning"`, and the extraBody moved to the + Qwen3.5-family recipe (0.7/0.8) with `chat_template_kwargs` + (a top-level `preserve_thinking` is ignored by vLLM). +- Deployer gap found pre-deploy: the spec-draft gate demanded a single + `model.safetensors` (the dflash layout) and would have rejected the + deployed MTP arm (4 shards behind an index); fixed to accept both + layouts, gated by bats. `z-lab/Qwen3.5-9B-DFlash`'s "vLLM PR #40898 + unmerged" card claim is FALSE - `vllm/v1/spec_decode/dflash.py` ships in + 0.30 (verified in-image); the drafter stays on disk as the alternative. +- Host state: +38 GiB fetched (MiMo BF16 17.53 GiB, Qwen3.5-9B 18 GiB, + DFlash 2.58 GiB), disk down to ~26 GiB free - reclaim pass due. gufo + rollback pinned at git tag `frank-dragoon-gufo-q4kxl-20261001`; engine, + Q4_K_XL payload and DFlash2 drafter all remain on the host. +- First deploy attempt #2 crash-looped the container 51 times: the serve + script handed vLLM the HOST model path while the tree is mounted at + `/models` inside the container — transformers rejected it as a malformed + repo id. Fixed by converting to the container path (the spec draft was + already converted; the target was not), pinned by bats. Also fixed the + deployer's own draft gate (same single-file bug as the serve script's). +- Reclaim (Frank approved): Swift-1.5 Q8_K_XL payload (30 G, refetchable via + `scripts/download-frank-dragoon-swift.sh`) and the orphaned flux-2-klein + tree under `models/checkpoints` (23 G; ComfyUI mounts + `comfyui-data/models`, not this) purged via + `~/.AGENTTRASH/20261001-frank-dragoon-reclaim/` — 52 G freed. KEPT the + 1.1 G `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` (it is the gufo rollback's + drafter, not a remnant). + ## 2026-09-30 - **frank-archon cut over to GLM-5.3-Flash W4A16 int4 — it serves, but NOT at the 35 tok/s target, and the shortfall is measured rather than inferred.** @@ -650,7 +1949,6 @@ - Rollback: commit `2339fae` (qwen FP8 + MTP4); the qwen payload and the R9V state dirs are intact on disk. - ## 2026-09-29 - **frank-dragoon tuning investigation: a measured NEGATIVE result, nothing changed.** Frank asked for iterative decode/prefill improvement at a 32k @@ -1166,1216 +2464,3 @@ - `/dankclaw/pihole/sysctl/99-dnsmasq.conf` — Repo source of truth for `ip_unprivileged_port_start=53`; `deploy_pihole` stages conflicting drop-ins, comments out the stale sysctl.conf line, and applies `sysctl --system` (the boot code path). - `/tests/caddy_quadlet.bats` (new) + caddyfile/deployment_flow/pihole seams — quadlet boot persistence, self-healing, network, env, and sysctl coverage. - `/serverAppConfigs-docs/Infrastructure/Caddy.md`, `/serverAppConfigs-docs/Infrastructure/Pi-hole.md`, `/serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md`, `/serverAppConfigs-docs/Dankclaw/Gitea.md`, `/serverAppConfigs-docs/Architecture/Deployment Flow.md` — quadlet runtime facts, the 2026-08-26 incident, the sysctl-conflict failure mode, canonical Caddy ownership. - -## 2026-10-01 — frank-archon RESTORED to fp8 3 x 262144 (bf16 boot fails) - -bf16 (--kv-cache-dtype auto) at 2 x 262144 did not stay up, with OR without ---expert-precise-memory: - -- with precise: NRestarts 1->3, each attempt ~6.7 min then - `Deactivated successfully` (API server drains after an EngineCore fatal - error) — cause not captured, the serve script rm'd the container. -- without precise (default 7.60 GiB overhead, arena 14.3 GiB): NRestarts - climbing, container `Exited (1)`, and the journal finally showed the real - error: - `RuntimeError: Worker failed with error 'Constraints violated - (L['query_start_loc'].size()[0])! You marked L['query_start_loc'].size()[0] - as dynamic but your code specialized it to be a constant (3)'` - — a torch.compile/dynamo dynamic-shape constraint violation, NOT an OOM. - `~/.cache/vllm` holds only `modelinfos` (16K) and each `podman run` gets a - fresh container filesystem, so a stale compile cache is ruled out: the - conflict is produced by this configuration's own compile path. - -Differences from the last shape measured serving (fp8 3 x 262144, MNBT 2048, -KV 6.1e9 -> 792,069 tokens / 3.02x / arena 6,248 / 3.59 GiB free): KV dtype -(fp8 -> bf16) and SEQS (3 -> 2). Both are candidates for the dynamo trigger; -`--enforce-eager` is the bypass that would sidestep the constraint path. - -Restored: fp8 3 x 262144, MNBT 2048, KV_BYTES 6,100,000,000, no precise flag. -Next: bisect bf16 vs SEQS in a window (bf16 + SEQS=3 isolates the dtype). - -## 2026-10-01 — frank-archon: --expert-precise-memory + bf16 = boot crash loop - -**Reverted.** The two changes Frank asked for were applied together and the -result crash-looped: `NRestarts` climbed 1 -> 2 -> 3, each attempt running -~6.7 minutes then logging `Deactivated successfully` (vLLM's API server drains -and exits 0 after an EngineCore fatal error, so `podman wait` returns 0; the -new `Restart=always` turned a silent death into a visible loop). - -The budget arithmetic explains it without needing the error text, because the -arena is DEVICE-resident (the fp8 3-window shape's plan predicts 28.27 GiB and -that is exactly what `rocm-smi` measured): - -| line | bf16 2 x 262144 + precise | fp8 3 x 262144 | -|---|---|---| -| non-expert weights | 6.28 | 6.28 | -| pinned pool | 6.80 | 5.68 | -| CUDA graphs | 0.55 | 0.55 | -| arena | 16.65 (precise: overhead 5.25) | 14.31 (default: overhead 7.60) | -| non-torch | ~1.45 | ~1.45 | -| **total** | **31.73 of 31.86 GiB** | 28.27 (measured) | -| free at boot | **~0.13 GiB** | 3.59 | - -`--expert-precise-memory` hands the 2.35 GiB compile/graph-capture headroom to -the arena, and bf16 simultaneously adds 1.08 GiB of pinned pool; together they -put graph capture ~0.13 GiB from the ceiling, where every transient allocation -dies. So the flag is dropped and bf16 2 x 262144 ships with the default -overhead: arena 14.31 GiB, ~2.4 GiB free at boot. - -**Lesson:** the registry's "compile headroom" is not slack to be reclaimed by -default — it is the budget for the phase that allocates last. Reclaim it only -by paying for it elsewhere (smaller pool, fewer sessions, or a shorter context). - -## 2026-10-01 — frank-archon: 2 bf16 windows + --expert-precise-memory - -Frank's final shape for the night. Two changes at once, both measured in -principle before the boot: - -- **KV is bf16 (`--kv-cache-dtype auto`), two full-size windows (2 × 262144).** - bf16 measures **13,839 B/token** (from the bf16 3×241664 boot: - 11,047,010,304 B -> 798,280 tokens), decomposing as **12,276 attention + - 1,563 mamba/side caches** against fp8's **6,138 + 1,563 = 7,701**. So - 524,288 tokens needs 7.256e9 B; pinned at 7,300,000,000 (~527,500 tokens, - 3,206 margin). Cost: the pool goes 5.68 -> 6.76 GiB per rank. -- **`--expert-precise-memory`.** The registry's default overhead for this - architecture is 7.60 GiB = the fitted 5.25 GiB (gfx1201 TP2 - nbt4096/seqs16/len262144) + 2.35 GiB of compile/graph-capture headroom. - The flag selects the fitted figure, which moves only the plan's `room` term - — arena 6,248 -> ~7,278 slots (~1,025 more, at 2.344 MiB/slot) — at the - cost of boot-time capture slack. The overhead formula adds 0.22 GiB per - 1024 nbt above 4096, so MNBT 2048 pays nothing. - -Budget note worth keeping: the arena is `(budget - non_expert_weights - -kv_reserve - graphs - overhead) // expert_bytes`, where `kv_reserve` is the -planner's OWN one-max-model-len reserve (2.48 GiB at TP2) and does NOT scale -with SEQS. The pinned `--kv-cache-memory-bytes` therefore does not shrink the -planned arena; it comes out of free VRAM instead. - -## 2026-10-01 — frank-dragoon: Q4_K_XL payload + 4 x 163840 sessions - -Frank's call. Payload swapped from ajgazin's **abliterated** Swift-1.5 -UD-Q8_K_XL (29.3 GiB) to **unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL** (16.35 GiB, -sha256 3f227079…b01e, vanilla/Apache-2.0 — the endpoint is no longer -uncensored). New fetcher `scripts/download-frank-dragoon-q4.sh`; the DFlash2 -drafter and its fetcher are unchanged. - -Shape: **`--sessions 4 --context 163840`** with unquantized KV. gufo has no -KV-quantization lever at all (`execution_policy.hpp` pins `kv_cache_storage` -to `kFp16`), so "unquantized" is inherent. - -KV arithmetic (why this fits where one 262144 session used to sit): -f16 KV costs **64 KiB/token** — 16 full-attention layers (the hybrid is -`full_attention_interval: 4` over 64 layers) x 4 KV heads x 256 head_dim x -2 (K,V) x 2 bytes; the other 48 layers are linear-attention with a -per-SEQUENCE fp32 state (~144 MiB), not a per-token cache. 4 x 163840 = -655,360 tokens = 40.0 GiB of KV, and each session is exactly 10 GiB. Budget: -weights 16.35 + KV 40.0 + drafter 1.07 + linear state 0.56 + runtime ~4 = -~62 GiB against the 114 GiB GTT. - -## 2026-10-01 — e2e gate flakiness (12-crawl) — do not panic at the banner - -`specs/12-crawl.spec.ts:23` ("every user open link reaches its app") failed the -deploy gate twice and then passed cleanly on a manual re-run: -- run 1 (deploy): `Expected "plex.franksdank.cloud"` / `Received "app.plex.tv"` - — Plex redirects its web UI to the hosted app; the same failure appeared on - the previous deploy too. -- run 2 (manual): `page.goto: net::ERR_ABORTED at https://www.franksdank.cloud/` - — the homepage answered `http=200` in 40 ms from the dev box at the same - moment, so this is a browser-level flake. -- run 3 (manual): **33 passed**, exit 0. - -So a red gate on this spec alone is NOT proof the fleet is broken, and the -deployer's `DEPLOY INCOMPLETE` banner can be raised by a flake. Re-run -`tests/e2e/run.sh` before treating it as real; the Plex redirect is the one -stable underlying issue and it lives in Plex's web-client/image config, not in -any repo service. - -## 2026-10-01 — frank-archon bf16 no-spec cutover - -- `/scripts/frank-archon.env` — bf16 no-spec cutover, FINAL shape after the - night ladder: MAXLEN **241664** x SEQS 3, KV_BYTES **10,184,000,000**/rank, - KV_DTYPE auto (bf16), MNBT 1024, GATHER CHUNKS=32/LANES=64, UTIL 0.98; - FRANK_ARCHON_SPEC deleted (no spec). Engine measured at the winner: - "KV cache size: 732,359 tokens, Maximum concurrency for 241,664 tokens per - request: 3.03x". -- **The night ladder (Frank's method).** 262144 OOMed at CUDA graph capture - (`Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB - allocated by PyTorch) and 245760 OOMed identically; **237568 PASSED** - (KV 718,606 tokens) and the +4096 bonus step **241664 PASSED and is - PRODUCTION**. 241,664 is max possible on this box at util 0.98 — the - model's native 262,144 does not fit with a 3-seq contract. Every rung was - booted under a transient systemd unit at uid dank with - `LimitMEMLOCK=infinity` (the rootless bench probe masked the production - memory environment) with full engine-log capture. -- **Util stays 0.98.** The earlier 0.90 edit was wrong — lowering - `--gpu-memory-utilization` shrinks the engine's own budget rather than - buying slack; Frank forbade it and the winning ladder ran at 0.98. -- `/scripts/serve-frank-archon.sh` — `--speculative-config` line removed; - `--max-num-batched-tokens` and the `-e R4D_LRU_GATHER_CHUNKS/LANES` - container-env passthrough added; shape header rewritten. -- `/tests/frank_archon_unit.bats` — seam re-pinned: bf16-KV-present, - spec-absent-by-measurement, gather-32x64-present, MNBT-1024 tests; the - qualified-shape and flags tests flipped to the new env keys. -- Overnight redo campaign (prefetch OFF everywhere) tabled in - `/serverAppConfigs-docs/LLM/frank-archon.md` - with the 2026-10-01 cutover section: gather 32x64 +18.7%, MNBT flat - 1024-4096, KV pool law (7,926 fp8 / 14,047 bf16 B/token), arena flat zone, - six-slot refusal, bf16 3x262144 boot failure root cause (CUDA OOM at - KV_BYTES 11.3e9; boots clean at 11.047e9 — captured from journald `-o json`). -- `/AGENTS.md` — fleet table frank-archon row rewritten to the bf16 shape. -- `/serverAppConfigs-docs/LLM/index.md` — fleet intro + mermaid node - updated out of the GLM era. -- **Client contract fixed (thinking + tools).** The live endpoint returned a - null `reasoning` field and dumped the thinking text INLINE in `content` - with a dangling `` (measured: `'We need answer…Brief.\n\n\n391'`) - — the malformed thinking block clients choke on — because the deployed argv - had no `--reasoning-parser`. The GLM rollback (`d91379d`) had dropped all - three contract flags and the qwen rollback inherited the gap. Restored from - `ecaad36`'s values, confirmed against this fork's registries: - `--reasoning-parser qwen3`, `--enable-auto-tool-choice - --tool-call-parser qwen3_xml`, `--default-chat-template-kwargs - '{"enable_thinking": true, "preserve_thinking": true}'`. The seam's - "NO tool dialect" test is flipped to a PRESENT test. -- **Final shape (Frank's call): 5 seqs x the full native 262144 window at - fp8 KV.** 6 x 241664 at fp8 died (`Tried to allocate 10.70 GiB`, 9.18 GiB - free) — six full-length contexts do not fit at any KV dtype — so the - contract moved to five FULL windows: 5 x 262144 fp8 PASSED on the first - rung, KV_BYTES 10,388,766,720/rank, measured **1,350,182 tokens / - 5.15x concurrency** (39,462 tokens of margin). Seam re-pinned; the bf16 - three-context shape (241,664, KV 732,359 / 3.03x, measured earlier the - same night) is the documented max at 3 contexts. -- Harness `/home/blank/.omp/agent/models.yml` — contextWindow back to 262144 - with the five-window fp8 shape (compaction gap 183,501 + 36,864 = 220,365 - <= 262,144) and - `preserve_thinking` moved from a bare top-level body field to - `chat_template_kwargs` (the form vLLM actually reads). -- **Runtime-survivability correction: five slots -> FOUR.** 5 x 262144 fp8 - passed every boot gate (KV 1,350,182 tokens / 5.15x) but left 0 bytes free - at runtime; a real client request (995-token prompt, `max_tokens=261149`, - sampling params straight from models.yml) killed Worker_TP1 with - `torch.OutOfMemoryError: Tried to allocate 50.00 MiB ... 0 bytes free` and - the engine shut down. Boot is not proof of runtime survivability — after a - knife-edge shape change, send a large-`max_tokens` request before calling it - done. FOUR slots x 262144 fp8 (KV_BYTES 8,311,013,376/rank) returns ~1.93 - GiB per rank for activations/graph capture. Frank's call 2026-10-01. -- `/scripts/frank-archon.service` — **Restart=always, not on-failure** (plus - `StartLimitIntervalSec=0` and `OOMScoreAdjust=-500`). A runtime worker fault - makes vLLM's API server exit CLEAN, so ExecStart returned 0, systemd logged - "Deactivated successfully", and on-failure NEVER fired: the endpoint stayed - down until a human noticed. An explicit `systemctl stop` still wins, so - bench windows and deploys are unaffected. -- **Final shape (Frank's call, 2026-10-01): 3 x 262144, pool at the FLOOR, - MNBT 2048, rest to the arena.** MAXLEN 262144 x SEQS 3 = 786,432 tokens of - contract; measured fp8 costs 7,698 B/token, so 6.054e9 B is the pool floor - and the deployed 6,100,000,000 B leaves a ~5.9k-token margin while handing - the remaining ~2.2 GiB (versus the 4-slot shape) to the expert arena — the - decode-critical cache the fork sizes from the remainder. MNBT 2048 costs the - arena nothing (the sweep measured 6,499 slots at every rung <= 4096). -- **DCP investigated and closed** (see the parallelism survey in the runbook): - `--decode-context-parallel-size 2` is rejected at boot — DCP for GQA requires - TP > the model's KV heads (2), i.e. TP >= 4, i.e. four GPUs. Not a tuning - knob on this box. -- Production bring-up: the shape boots and serves on :7770 — engine log - verified (KV cache size, concurrency, MNBT 1024, gather 32x64) plus live - contract probes (reasoning split, tool_calls). Rollback target for the - 2026-09-29 fp8+MTP4 shape: git tag `frank-archon-fp8-mtp-20261001`. - -## 2026-10-01 — frank-dragoon cutover: gufo -> vLLM TheRock, MiMo-V2.6 BF16 + cross-model MTP - -- **Question that started it:** can a Qwen3.5 MTP drafter attach to - frank-zealot, and how does vLLM run the FULL BF16 MiMo checkpoint? Answer 1: - not on frank-zealot — SGLang reads the MTP head from the TARGET checkpoint - and MiMo ships ZERO `mtp.*` tensors (base and INT8 alike). Answer 2: the - digest-pinned `kyuz0/vllm-therock-gfx1151` image (vLLM 0.30 / torch 2.11 / - TheRock ROCm 10) boots BF16 MiMo on the gfx1151 iGPU in ~200-240 s, and - vLLM reads the draft from a SEPARATE path — so cross-model MTP from - `Qwen/Qwen3.5-9B` (the distill's source, identical 4096/248320/32-layer - geometry) works. Frank's call: ship that as frank-dragoon. -- **Bench window (frank-dragoon stopped, one session, temp 0, 28,647-token - prefill / 512 decode):** base 17,876 t/s prefill / 13.1 t/s decode (13.1 - is ~94% of the LPDDR5x BF16 roof - decode here is bandwidth, not a bug); - +DFlash-8 7,976 / 17.4 (acceptance 59%/26% per position); +MTP-2 7,380 / - 19.8 (71%/44%, token rate 57.4%). Deployed MTP-2: +51% decode. Both spec - arms pay ~55-59% of prefill. `num_speculative_tokens` beyond 2 unmeasured. -- **Measured numerics law:** spec arms do NOT reproduce the no-spec greedy - text (bf16 batched verification reorders reductions; near-tie argmaxes - cascade). vLLM spec decode is distribution-preserving, not - bitwise-greedy-preserving. -- **Client-contract delta wired:** vLLM emits `reasoning`, not - `reasoning_content` - `models.yml` frank-dragoon block now sets - `reasoningContentField: "reasoning"`, and the extraBody moved to the - Qwen3.5-family recipe (0.7/0.8) with `chat_template_kwargs` - (a top-level `preserve_thinking` is ignored by vLLM). -- Deployer gap found pre-deploy: the spec-draft gate demanded a single - `model.safetensors` (the dflash layout) and would have rejected the - deployed MTP arm (4 shards behind an index); fixed to accept both - layouts, gated by bats. `z-lab/Qwen3.5-9B-DFlash`'s "vLLM PR #40898 - unmerged" card claim is FALSE - `vllm/v1/spec_decode/dflash.py` ships in - 0.30 (verified in-image); the drafter stays on disk as the alternative. -- Host state: +38 GiB fetched (MiMo BF16 17.53 GiB, Qwen3.5-9B 18 GiB, - DFlash 2.58 GiB), disk down to ~26 GiB free - reclaim pass due. gufo - rollback pinned at git tag `frank-dragoon-gufo-q4kxl-20261001`; engine, - Q4_K_XL payload and DFlash2 drafter all remain on the host. -- First deploy attempt #2 crash-looped the container 51 times: the serve - script handed vLLM the HOST model path while the tree is mounted at - `/models` inside the container — transformers rejected it as a malformed - repo id. Fixed by converting to the container path (the spec draft was - already converted; the target was not), pinned by bats. Also fixed the - deployer's own draft gate (same single-file bug as the serve script's). -- Reclaim (Frank approved): Swift-1.5 Q8_K_XL payload (30 G, refetchable via - `scripts/download-frank-dragoon-swift.sh`) and the orphaned flux-2-klein - tree under `models/checkpoints` (23 G; ComfyUI mounts - `comfyui-data/models`, not this) purged via - `~/.AGENTTRASH/20261001-frank-dragoon-reclaim/` — 52 G freed. KEPT the - 1.1 G `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` (it is the gufo rollback's - drafter, not a remnant). - -## 2026-10-02 — frank-dragoon cutover: vLLM MiMo -> ROCmFPX Q8_0_ROCMFPX_AGENT + MTP - -- 2026-10-02 — frank-dragoon: vLLM MiMo → ROCmFPX `Q8_0_ROCMFPX_AGENT` - payload + MTP from the repo's own bundled head, one session at the native - 262,144. See [[frank-dragoon]]. -- The runtime is **forced by the file**: tensor type 103 - (`GGML_TYPE_Q8_0_ROCMFPX`, ftype 115) is unreadable by stock llama.cpp, - gufo and every other loader on the fleet — only the kingjones30/ROCmFPX - fork can read it, so the engine is pinned to commit - `36e9acd40e10a87cd3c3ef8ec734668757dc8520` and built by - `scripts/build-rocmfpx.sh`. The AGENT routing variant was chosen over the - card's plain Q8_0 because it keeps 187 output-head tensors at plain Q8_0 - instead of 2, which measured better MTP acceptance. MTP was the whole - performance story of the shape (roughly +120 % decode, ladder on the - runbook). -- Rollback: git tag `frank-dragoon-gufo-q4kxl-20261001`; the retired - vLLM-era fetchers moved to `old/`. Runbook [[frank-dragoon]]. - -## 2026-10-02 — frank-zealot: Q5_K_M, then three contexts + vision, sized by two boots - -- 2026-10-02 — frank-zealot: Q5_K_M payload, then three contexts + vision - in one day, sized by two boots (`CTX_SIZE=540672 PARALLEL=3`, f16 K + - q8_0 V under flash attention, `--mmproj`). See [[frank-zealot]]. -- **The pool came from two boots, not a formula.** Differencing the boots - prices the non-KV floor — and prices the vision tower at **~1.4 GiB for - a 918 MB file**: the encoder's own buffers ride with it. The first 3-slot - deploy booted healthy but left less than the ordered 2–3 GiB band free, - which is why the seam now prices non-KV at the measured value and - REJECTS that shape. -- Three contexts cost window, and that is the tradeoff Frank was told: - 180,224 tokens per conversation, not the native 262,144; a longer prompt - is a hard `400 exceed_context_size_error`, so `models.yml` follows the - slot, not the card. -- **Vision contract gotcha:** `/v1/chat/completions` ignores the legacy - top-level `images` field — that request answered "Brown." for the same - red image, i.e. it generated from text alone. Only the `image_url` - content part reaches the model. `/completion` accepts `images` but this - build still requires `messages`. -- Deployer gates on the projector too (`LLM_SMALL_MMPROJ`), so a missing - mmproj cannot silently ship a blind endpoint; `MMPROJ_FILE=` - (explicitly empty) in the env file is the documented text-only opt-out - and returns the tower's ~1.4 GiB to KV. -- Reclaim staged, **not purged, awaiting approval**: - `~/.AGENTTRASH/20261002-retired-llm-payloads/` of superseded payloads. - Runbook [[frank-zealot]]. - -## 2026-10-02 — frank-archon shape D: 3 x 200704, fp8, pool sized for 2 contexts - -Frank's order: shrink frank-archon to a 200704-token window with about two -resident full-length contexts, cut the KV pool, and drop MNBT to 1024. The -first attempt (`30904ba`, 2 x 200704 + SEQS=2) **crash-looped**; the revert -went live as `e460bbb`, and the shape was then re-applied one law at a time on -top of the proven-good base, root cause first. - -- **ROOT CAUSE: `--max-num-seqs 2` cannot compile this model. It was never - OOM, never MNBT, never the KV dtype.** Four single-variable boot probes - against the then-deployed shape (production stopped for each, candidate - booted AS the service): 262144 x SEQS 2 fails at BOTH MNBT rungs with - `torch.fx.experimental.symbolic_shapes.ConstraintViolationError: - Constraints violated (L['query_start_loc'].size()[0]) ... specialized it - to be a constant (3)`, raised from AOT `build_guards(strict_error=True)` - during `determine_available_memory -> profile_run -> _dummy_run`, ~7 min - in; SEQS 3 is healthy at both rungs. `model_runner.py` sizes that input - buffer `max_num_reqs + 1` and `_dummy_run` sets `num_reqs = - min(max_num_batched_tokens, max_num_reqs)`, so its width is **SEQS + 1** - for ANY `MNBT >= SEQS` — 3 at SEQS 2, exactly the constant in the - message; MNBT only moves the compile RANGE, which is why the identical - error appears at both rungs. `--enforce-eager` bypasses the path but - drops CUDA graphs. The 2026-10-01 bf16 2 x 262144 failure was this same - error class; its message named "KV dtype and SEQS" together and nobody - bisected, so the dtype was blamed — it was a red herring. -- **THE ARENA LEVER IS `max_model_len`, NOT `--kv-cache-memory-bytes`.** - The planner sizes expert cache as `budget - non-expert - KV reserve - - graphs - overhead`, and the KV reserve is registry-priced at **9190 - B/token x max_model_len**, independent of the pool override: d(reserve) - equalled d(arena) exactly, and the multi-GB pool cut bought **zero** - arena slots. The previous session's premise — "the pool cut becomes - arena, ~7,500 slots" — was wrong; the pool cut is a runtime-HEADROOM - lever (the shape it replaced died with `0 bytes free` on a real - 995-token request). -- **Final shape (option D):** the argv keeps the only slot count that - compiles; the POOL caps resident conversations (ledger on - [[frank-archon]]). Measured live, one prediction corrected: the arena - prediction landed exactly, the concurrency one did not — **the per-token - KV price is shape-dependent** (7,701 B/token at MAXLEN 262144 vs **7,892** - at 200704, ~2.5 % dearer). Never extrapolate a concurrency figure across - geometries. Smoke probes confirmed the 3 slots real, native `tool_calls`, - and that `reasoning_effort` accepts only {xhigh, medium, low}, answering - 400 for anything else. -- **Harness contract followed the window:** `models.yml` `contextWindow` - 262144 -> **200704**; side effect: frank-archon's compaction request fits - inside a frank-zealot slot again. A comment-only follow-up `295d578` was - deployed so the per-service revision marker sits at HEAD. Rollback for - the pre-shape-D fp8+MTP4 era stays git tag - `frank-archon-fp8-mtp-20261001`. Runbook [[frank-archon]]. - -## 2026-10-02 - plex: the readiness gate probed HTTP against a TLS-only service - -Found by accident, and it had been latent since the secure-connections posture -was set: `deploy_plex` gated on `curl -fsS http://127.0.0.1:32400/identity`, -which **cannot** succeed. PMS drops plain HTTP off-loopback - the probe's -connection arrives on the container's eth0 - so Plex accepts the TCP connection -and then answers with nothing, and curl reports "Empty reply from server" (52). - -- Trigger: a bare `./scripts/deploy.sh dankserver` (no `--service`) drains - the whole pending backlog, not just the service you were thinking about. - plex's per-service revision had been behind long enough that this gate - had never run against the current posture; it burned its full 600 s and - failed the run, and every service after plex in the order was left - undeployed. -- **Plex was healthy the entire time.** `https://127.0.0.1:32400/identity` - returned 200 with the MediaContainer XML in 44 ms while the gate was - still timing out; the in-container plain-HTTP path also answers 200 — - loopback is the one path PMS serves in the clear, which is exactly why - the host-side probe could not see the difference. -- **The repo already knew this law, in a different file:** - `tests/caddyfile.bats` pins the plex vhosts to a TLS upstream with the - comment "PMS drops plain HTTP off-loopback". The deployer's gate never - got the same treatment. Fixed to `curl -fsSk --max-time 3 - https://127.0.0.1:32400/identity` (`-k`: PMS serves its own self-signed - cert on 32400). Re-deployed `39f0e0d`: plex recorded deployed, e2e - 33/33, frank-archon untouched (`health=200`, NRestarts=0). -- **Operating lesson: `deploy.sh ` with no `--service` is a - fleet-wide drain.** It restarts every service whose recorded revision is - behind HEAD and whose paths changed since. To touch one service, name it. - -## 2026-10-02 — frank-archon shape E: `--expert-precise-memory` (+1,026 arena slots), a mounted compile cache, and the P2P answer - -Shape D left one question open — the plan's overhead term was reserving 2.35 GiB -of compile headroom the boot never used — and the answer to Frank's free-VRAM -question arrived together with a boot-cost bug that had been costing 181 s on -every restart since the cutover. - -- **`--expert-precise-memory` is real and it fits (option A).** Boot-probed on - the shape-D base with production stopped, the candidate booted AS the service - (`FRANK_ARCHON_ENV_FILE=/tmp/probe-b.env bash scripts/serve-frank-archon.sh`, - trap-guaranteed restore), the numbers read off the engine's own plan lines: - - | plan line | shape D | shape E | - |---|---|---| - | `runtime overhead (non-torch + buffers + activation)` | 7.60 GiB | **5.25 GiB** (`precise (gfx1201 TP2 nbt4096/seqs16/len262144)`) | - | `expert cache space` | 14.87 GiB | **17.22 GiB** | - | slots | 6,496 | **7,522** (+1,026, +16 %) | - | `swap region` | — | 13.7 GiB / 5,986 slots, 133-134 per layer | - | boot | — | **healthy in 420 s**, no OOM | - - The flag also prints `'expert_precise_memory': True` in `non-default args`, so - a silent argparse drop cannot hide. It was refused on 2026-10-01 only because - the 2 x 262144 bf16 shape was already 0.13 GiB under the ceiling — there the - flag was the last stack over the line, not the cause. -- **The pool was NOT cut (option A) because the pool is not an arena lever.** - A 2.2e9 arm was probed in the same window and reports `277,686 tokens, - 1.38x` — measured, rejected as the shipped value, kept as the conservative - fallback. With 3.15e9/rank the endpoint keeps shape D's 1.99x concurrency and - gains the arena, which is the shape D follow-up had actually been looking for. -- **THE COMPILE CACHE WAS BEING THROWN AWAY ON EVERY RESTART — 181 s each.** - Measured: a fresh container has **no `/root/.cache` at all** (`No such file or - directory`); after our boot it holds `/root/.cache/vllm` at - **1,154,714,333 B in 4,665 files** plus `/root/.cache/comgr` at 229 MB, inside - the ephemeral layer, and `podman rm -f` (every deploy, restart, reboot) - discards them. The engine names the loss itself: `Compiling a graph for compile - range (1, 1024) takes 173.19 s` ... `torch.compile took 181.20 s in total` ... - `init engine (profile, create kv cache, warmup model) took 217.26 s - (compilation: 181.20 s)`. The serve script now mounts - `FRANK_ARCHON_CACHE_DIR` at `/root/.cache` — the **parent**, so the vLLM - AOT/torch_compile tree (`VLLM_CACHE_ROOT`, default `/root/.cache/vllm`) and - the comgr JIT cache both persist, and nothing is hidden because the image has - no such directory. Same class of fix as the recipe's - `/cache/{triton,vllm,inductor}` volumes; the compile RANGE is part of the - cache key, so an MNBT change costs one recompilation, not a permanent tax. -- **`GPU_MAX_HW_QUEUES=2`** (HSA hardware queues per GPU) added at Frank's - direction. Verified a real knob rather than a no-op — the literal is read from - `/opt/rocm/core-10.0/lib/libhsa-runtime64.so` — and shipped deliberately - alongside the 32x64 gather geometry. **Not A/B measured.** -- **P2P: the hardware supports it, and the engine is already using it.** The - live log shows `r4d all-reduce INSTALLED (rank=0 max=49152KB finegrained=T…)` - on BOTH ranks and `Using ['R4D_AR', 'PYNCCL'] all-reduce backends` — R4D's - P2P path, with RCCL as fallback, measured at **45.274 GB/s bidirectional**. - Root ports `00:01.1`/`00:01.3` both report ACS `ReqRedir+`, so P2P hairpins at - the root complex; there is no XGMI. The old FAIL predates the 2026-09-20 board - swap. Kernel support confirmed: `CONFIG_HSA_AMD_P2P=y`, `CONFIG_PCI_P2PDMA=y`, - `CONFIG_HSA_AMD=y`, `CONFIG_ZONE_DEVICE=y`, `iommu=pt` on the cmdline. -- **Recipe diff (Frank's 3 x R9700 compose vs this podman run), ranked:** - - *Acted on:* persistent compile caches (above — the one real operational - gap), and `GPU_MAX_HW_QUEUES`. - - *Retracted:* pinned `--compilation-config cudagraph_capture_sizes` was on my - list; measured, it is **not** a gap here — capture is 3 PIECEWISE + 2 FULL - graphs, `Graph capturing finished in 3 secs, took 0.31 GiB` (0.11 GiB on the - lighter rank). No 15-52-size ladder to cut. - - *Not acted on — measured candidates, none shipped (no bench window; these - are CPU-side and our decode is gather-bound, so a spin-wait stealing cores - from the gather threads is plausible but unproven):* `OMP_NUM_THREADS=8` - (defaults to all 16 logical CPUs on this 7700X), `HSA_ENABLE_INTERRUPT=1`, - `HSA_ENABLE_MWAITX=1`, `HSA_ENABLE_SDMA`, `HSA_NO_SCRATCH_RECLAIM`, - `HSA_ENABLE_IPC_MODE_LEGACY`. All six HSA names verified real — the literals - are read out of `/opt/rocm/core-10.0/lib/libhsa-runtime64.so` — so they are - knobs, not no-ops; any of them needs its own A/B before shipping. - - *Deliberate divergence, keep ours:* no MTP (the recipe has it; measured - fastest here), `qwen3_xml` vs `qwen3_coder` (ours verified live), no - `--override-generation-config` (server-side sampling defaults), published - port + Caddy rather than `network_mode: host`. - - *Non-gaps:* `shm_size 32gb` (ours is `--ipc=host`), `memlock -1` (same), - `SYS_PTRACE` (same), `/dev/kfd` + `/dev/dri` (same), `--enable-chunked-prefill` - (V1 default), `plecache` volume (the flag is not even passed), and - `ROCR_VISIBLE_DEVICES=1,2,3` vs our `HIP_VISIBLE_DEVICES=0,1` — verified - equivalent here: `rocm-smi --showid` puts the two R9700s at GPU[0]/GPU[1] and - the 7700X iGPU at GPU[2], so our indices are the right two cards. -- **Harness note, because it cost two probe attempts:** a probe that reported an - instantaneous candidate death was a logging artifact of the inline SSH script, - not a crash — the container and its serve script must be polled by recorded id - and their logs dumped on the exit path, or a failure looks like a bad flag. - The flag was fine both times. -- Live: `health=200`, `NRestarts=0`, arena 17.22 GiB / 7,522 slots, KV 398,658 - tokens / 1.99x. Docs: [[frank-archon]]. -- **Deployed `1ac4d48`, e2e 33/33, and the live plan matched the probe exactly:** - `'expert_precise_memory': True`, `runtime overhead … 5.25 GiB`, `expert cache - space 17.22 GiB / 7522 slots`, `swap region 13.7 GiB / 5986 slots`, `GPU KV - cache size: 398,658 tokens … 1.99x`, pool 3,150,000,000, MNBT 1024. The - precise overhead and the arena growth are now production numbers, not a - probe's projection. -- **The compile-cache fix was then proven on the same endpoint, two consecutive - boots of the deployed shape:** cold `Compiling a graph for compile range - (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s`; warm `Directly load - the compiled graph(s) for compile range (1, 1024) from the cache, took - 2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile, - create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)`. The host - dir held 1.4 GB in 4,707 files after the cold boot. **Time to `/health` is - dominated by the 172.8 GiB payload load** (the warm boot still needed 490 s), - so the honest claim is ~184 s of inductor/AOT build recovered per restart — - not a proportional boot cut. - -## 2026-10-11 — frank-archon: unquantized (bf16) KV at 2 × 200,000 - -- 2026-10-11 — frank-archon: **unquantized (bf16) KV at 2 × 200,000** on - exact attention state, with pool, slab and host pool rebalanced around - the bigger cache. See [[frank-archon]]. -- Engine door: `--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the - whole cache, so llama.cpp's exact-K / quantized-V pair (what frank-zealot - runs) has **no radiance equivalent**. -- `--max-num-seqs 3 → 2` booted clean: the fleet's "seqs 2 cannot boot" law - is the retired tcclaviger vLLM fork's torch.compile `query_start_loc` - constraint, not radiance's — radiance booted seqs 2 in its fp8-native era - and the sweep had 2 and 3 tied inside noise. -- Harness law re-learned: radiance puts the thinking block in - `reasoning_content`, so a probe that reads `content` on a small budget - sees empty text with `finish_reason: stop` — a healthy server scored 0/2 - until the probe set `chat_template_kwargs.enable_thinking=false`. Tool - committed as `scripts/probe-frank-archon-kv.py`. -- The fp8-vs-bf16 control arms and the open keep-bf16-or-take-cheap-fp8 - call are recorded on [[frank-archon]]; the 2026-10-13 section below (in - file order above) closes that call in favour of bf16. - -## 2026-10-15 — frank-archon: publisher iq4r payload swap (MODEL_FILE-only) - -- 2026-10-15 — frank-archon: publisher iq4r payload swap, `MODEL_FILE`-only - — `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`, the engine author's - published rad-convert of the OFFICIAL Qwen3.8-Flash-Next, **no other - argument changes**, mindful the operator session runs on the endpoint - being swapped. See [[frank-archon]]. -- Disk: Frank authorized one `rm` — the spent BF16 source and the - two-era-old fp8-native `.rad` left `/home/dank/models`; fp8-native is now - rebuild-only (fetch + build scripts); rollback for THIS swap is the - previous `.rad`, on disk, by reverting the commit. -- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes + - Hub LFS sha256, `--verify-only` default) as a transient unit. The cutover - **self-severed by design** — a detached `deploy.sh` survived the session - death when frank-archon restarted; engine healthy in ~100 s, and the - prefix tier RECOVERED across the swap (arch+tokenizer unchanged). -- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw - rebooted 01:41 and the gluetun boot-race dependency-failed `searxng`, - `marreta`, and `marreta-browser` (jobs consumed while gluetun's start job - retried; `Restart=always` never armed because the services never ran). - Cascade restarted; skill codified - (`gluetun-dependent-boot-race-recovery`). frank-archon's own health gate - passed every deploy. - -## 2026-10-15 — authentik: two more human accounts (Dankclaw) - -- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, - no repo objects**, each `@franksdank.cloud` in the human-user - template shape (password via `set_password` on stdin only — never argv, - so nothing lands in `ps`), verified through the exact PasswordStage code - path; zero `proxysession` rows for either, so no claim purge was due. -- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user census - extended with pk 22–23. - -## 2026-10-17 — frank-zealot: Bonsai-2-27B PQ2_0 trial + cutover (2 × 200,704, q8_0 KV) - -- 2026-10-17 — frank-zealot: **Ternary-Bonsai-2-27B (PQ2_0) cutover**, two - slots at 200,704 over q8_0 K + q8_0 V on the RTX 3090, fetched by - `scripts/download-frank-zealot-bonsai2.sh` (exact bytes + sha256, - `--verify-only`). See [[frank-zealot]]. -- Runtime door: the file is `file_type 141` with 402 tensors of custom GGML - type **142** plus `prism.hadamard.*` metadata — stock llama.cpp refuses - it, so the PrismML vendor build is **forced, not chosen**; the vendor - build serves q8_0 V under flash attention, so the source rebuild with - `GGML_CUDA_FA_ALL_QUANTS=ON` was cancelled as unnecessary. -- Trial-window law cost one boot: the scratch env invented - `KV_CACHE_TYPE` where the serve script reads - `CACHE_TYPE_K`/`CACHE_TYPE_V`, so the script's `:=f16` default silently - won and every arm booted f16. A silently-defaulted knob turns a whole - trial window into a different experiment. -- New fleet laws recorded (AGENTS.md + runbook): the mmproj's worst-case - VRAM is NOT inside the engine's fit projection; the first request after a - boot is a warmup artifact; a 64-token thinking budget makes a thinking - model look blind; the Bonsai-2 template reads `reasoning_effort` - (xhigh/medium/low, raises otherwise) so the harness effort fold is - load-bearing on this endpoint, and it reads `preserve_thinking`. - Recurrent DeltaNet state bills ~150 MiB **per slot** — pool arithmetic - alone misses it. -- Base alignment: **Frank chose the OFFICIAL censored payload** — publisher - provenance over refusal-freedom. The 9B it replaced was uncensored, so - the homepage card and its warning contract were re-keyed: zealot's card - now carries the base-alignment notice, the shared legal warning belongs - to dragoon alone. The CRACK re-export (byte-identical geometry per header - census) stays on disk as the one-line `MODEL_FILE` fallback if refusals - bite. -- Two live-path stragglers the cutover had to catch: - `serve-frank-zealot.sh` still DEFAULTED to the retired payload's paths - (a defaulted run would have served the old model — now re-keyed to the - shipped trio), and the OpenWebUI `model` row's `meta.description` is live - DB state (re-keyed with a DB backup; no `open-webui` restart forced — - forcing it logs every family session out for one cosmetic string). - -## 2026-10-17 — dankserver disk purge: Swift w4dn8 rollback + orphan caches (docs re-key) - -- Ask: "dump the swift rollback and the GLM GGUF — make sure the model - caches and agenttrash are empty as well." Preflight first: `lsof` empty - on both payloads, the serving trio names only the live path, and a podman - layer census proved no deleted path backs an image container (the 121 GB - layer is the ENGINE image, not a model blob). Staged, then purged with - Frank's approval: the two retired payloads, derived kvcache tiers, stale - compile caches, and every `.AGENTTRASH` on the host. -- Recovery: the retired w4dn8 rollback payload is byte-exact on the Hub — - rollback is now a **re-download, not a local file**. Corpora (grams, - kld-ref, calibration JSONL) deliberately KEPT: they are the re-convert - and KLD-gate inputs. -- Post-checks: frank-archon active and `/health` ok, serving payload - byte-size intact, the live kvcache tier untouched, no process references - a deleted path. -- Docs re-keyed: AGENTS.md fleet row + frank-archon section + both runbook - index rows; the live runbook's Rollback rewritten (HF re-fetch procedure — - the fetch script's REPO/RAD_NAME/STAGE/DEST/SIZE/SHA256 constants are - hardcoded, so retargeting at the mirror is a script edit); the retired - runbook's header corrected (was still "LIVE", never updated at - retirement). - -## 2026-10-17 — frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool - -- 2026-10-17 — frank-dragoon: cutover to the native v2 abliterated halogen - payload with the card's memory-saving KV pool; census first: the n-gram - table was ALREADY disk-backed and the only real lever was the KV pool. - See [[frank-dragoon]]. -- **Filename trap caught before the fetch:** this repo name carried two - different artifacts 32 bytes apart (the writer-transplant era's file vs - the native conversion). Byte-exact gate + LFS sha256 in - `scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the - peonist n-gram/vision/tokenizer trio — the release publishes only the - main file, and a hardlink is a regular file, so the no-symlink serve gate - passes without a 47.7 GiB copy. -- **Companion risk adjudicated by the engine, not by argument.** The card - does NOT establish the stock peonist companions as matched to a native - conversion and publishes no source-matched pair, so the A/B ran the - checkpoint tools on a stopped service under one knob set - (`scripts/census-frank-dragoon.sh`) plus a behaviour battery - (`scripts/battery-frank-dragoon.py`): two sound models disagreeing on - content, not a corrupt forward pass. The boot window ran through a - transient `FRANK_DRAGOON_ENV_FILE` drop-in — never a scratch port (one - iGPU, one server) — with the stop+mask law and a trap-guarded restore - (`scripts/trial-frank-dragoon.sh`). Masking fails on a relinked unit - file, as documented; stop alone was enough. -- Cost of the swap, stated plainly: two MAXIMUM-length conversations no - longer share the endpoint, the finetune's perplexity edge is gone, and - its reasoning came out compressed ~1.9×. Bought: RAM back as page cache - for the disk-backed n-gram table, faster cold prefill, clean retrieval, - and the base checkpoint itself instead of the finetune. The seam's pool - law relaxed from `pool == slots × ctx` to `ctx ≤ pool ≤ slots × ctx`. - -## 2026-10-17 — frank-dragoon: ceiling and pool cut to 200,704; crawl reset fix - -- 2026-10-17 — frank-dragoon: context ceiling and KV pool cut to 200,704 - (49 × 4096, the per-slot rung frank-zealot runs; nothing in the fleet - needed the bigger window). See [[frank-dragoon]]. -- Harness moved WITH the ceiling (400-zone law): `models.yml` - `contextWindow` = ceiling − `maxTokens`; compaction fires at 85 % of the - window, and the 180k compaction target of the 262,144 era is unreachable - because the window itself is smaller. Seam re-keyed to prove the - invariant at the new pair — `ctx ≤ pool ≤ slots × ctx` and - `harness window ≤ ctx − maxTokens` — so a future raise of one without the - other fails the gate. -- Probes on the live endpoint: over the ceiling → `400 … tokens over` in - 0.86 s — preflight rejects before any prefill, so probing costs the - endpoint nothing. -- Deploy verdict: the endpoint dispatched healthy (`/health` 200), but the - `[3/3]` Playwright gate failed on `12-crawl` and the deployer correctly - said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was Plex - Web, signed out in the `qa-e2e` profile, running its own sign-in flow, - which broke the spec two different ways: (1) it redirects the top frame, - so Chromium cancelled the crawl's in-loop `page.goto("/")` with - `net::ERR_ABORTED` — and that reset sat outside the `try`, so one host - aborted the whole crawl; fixed with `backHome()`, which re-issues on that - one error class. (2) The same hand-off leaves the browser on - `app.plex.tv` (Plex's hosted surface for a session-less client), so the - `hostname == target` law could never settle; fixed with an explicit - `HANDOFFS` landing list. Plex itself was healthy throughout, and the - guard contract is still asserted for every vhost. Both symptoms recorded - as gotcha 6; the gate after both fixes went 33 passed with the crawl - green on the first attempt. - -## 2026-10-18 — issue #3 research: AMD PerfOpt retires the reason for `amd_iommu=off` - -- Ask: does a recent kernel change remove the IOMMU translation penalty - that justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the - XDNA2 NPU come back? Answered and documented; **no host change** at the - time — the fix is not installable from the Ubuntu 26.04 archive. -- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13): merged - via the iommu tree 2026-09-24, first release **Linux 7.4**. It stops - GPA→SPA translation for the *privileged iGPU* only and leaves the IOMMU - fully armed for every other device, NPU included. Its only consumer - requires `AMD_IS_APU` **and the identity domain** — so PerfOpt rides on - top of `iommu=pt`, it does not replace it. -- **Three premises of the issue corrected.** (1) The 13–16 % is the - *vendor's* A/B against *their* previous boot line — our record copied the - figure on 2026-09-11 without ever measuring here. (2) The NPU does **not** - have to wait for 7.4: `iommu=pt` enables the IOMMU, which is all - `err=-19` asks for (`amdxdna` binds via `iommu_sva_bind_device()` into its - own SVA domain, independent of the default domain type). (3) The 460 - tok/s baseline is stale, so the window must measure `off` vs `pt` locally - and inherit no number. -- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and the - **Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % - on Strix Point / Krackan laptops) — the modest arm is the one that - describes this box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU - regression disables SVA outright. -- Self-correction inside the same session: the first pass claimed - `iommu=pt` alone already avoids the cost. It does not — an identity - attach keeps the page walk and enables ATS/PRI/PASID/GCR3, so `pt` - recovers **most** of the gap and PerfOpt is the last part of it. The - posted comment (Gitea issue #3) was edited to the corrected text. -- Artifacts: a new section in `dankclaw-iommu-translation-off.md` - (front-matter re-keyed to "vendor-claimed, never measured locally"), - `frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a - middle ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, and - a 4-step maintenance-window procedure that measures locally instead of - inheriting the vendor number. Nothing deployed — no service was touched. - -## 2026-10-18 — boot line flipped: Dankclaw now boots `iommu=pt` - -- Window executed 01:24–01:35 on Frank's go-ahead, with the A/B the - research demanded. Provenance first: `/etc/default/grub` had never been in - the repo while the IOMMU line was the most consequential kernel decision - on the box, so it is now tracked at `dankclaw/host/grub` (fetched - byte-exact) and moves only through `scripts/set-dankclaw-bootline.sh` — - diff, install 0644 root:root, `update-grub`, **verify the default - `grub.cfg` entry**, reboot on `--reboot` only. The deployer plans nothing - under `dankclaw/host/`: kernel state is not a container restart. -- **Verdict: identity DMA costs nothing on this host.** Byte-identical - traffic, +2.0 % cold prefill going OFF→PT; the 13–16 % that justified - `amd_iommu=off` for five weeks never existed here. Decode's apparent - −9.7 % was instrument noise: three repeats of the same request on the - same boot spread 31.8–47.8 t/s. -- **The XDNA2 NPU is back:** `amdxdna` bound `0000:c6:00.1`, - `/dev/accel/accel0` `crw-rw---- root:render 261,0` with `dank` in - `render`. No endpoint deployed — revival needs a quadlet with - `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`. -- Deliberately NOT done: the `pp_dpm_fclk` level-7 pin (it changes iGPU - timing for every workload and nothing co-runs on the NPU yet) and any - PerfOpt work (kernel `7.0.0-38` has zero `perfopt` symbols; the 26.04 - archive caps at `7.0.0-38.38`). -- Gates after boot: 0 failed units, 32 IOMMU groups (31 identity + 1 DMA = - the Genesys Logic SD reader, a kernel quirk), interrupt remapping on; - frank-dragoon's ledger byte-for-byte the same shape and Gitea / Caddy - edge / Grafana answered 200. New tool `scripts/stage-frank-dragoon-eval.sh` - rebuilds the A/B corpus byte-identically — run it through `bash -lc`, `uv` - is in `~/.local/bin`. Docs re-keyed: the decision record - `dankclaw-iommu-translation-off.md` (new § *execution*), `frank-npu.md` - (precondition satisfied, endpoint still absent), AGENTS.md key lesson. - -## 2026-10-18 — frank-zealot: Bonsai-2-9B PQ2_0 with the MTP head (3 × 262,144) -- 2026-10-18 — frank-zealot: 27B PQ2_0 → **Ternary-Bonsai-2-9B PQ2_0 with - the MTP head**, runtime moved to the PrismML vendor fork build **10777** - because the MTP arm needs the build that pairs MTP with a quantized V - cache. See [[frank-zealot]]. -- The win is the **window, not the quant**: every conversation goes back at - the trained 262,144 (the 27B era sold each slot 200,704), and the fleet's - widest context makes :7771 the **compaction hub** — `models.yml` - `contextWindow` 200704 → 262144. -- **MTP armed, then OFF by measurement.** The head is in the file and the - fork loads it, but the draft context bills its own **f16** KV over the - WHOLE pool, and at `UBATCH_SIZE=2048` that squeeze **halved decode**. The - documented MTP shape (`SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024`) was - declined — the decode gain is not worth spending the free card on it. The - arm is recorded in the env header, not left to rediscovery. -- Two bugs the trial caught, both fixed in the repo: - `${SPEC_PRESENCE_PENALTY:=0.0}` was a *default-assignment*, so the env - value always won and the documented "MTP runs without - `--presence-penalty`" arm was unreachable — now `${VAR=0.0}`, which lets - `SPEC_TYPE=draft-mtp` drop the flag. And `up{job="frank-zealot"}` had read - 0 for the **entire retained Prometheus history**: llama.cpp answers - **501** on `/metrics` unless `--metrics` is passed, so the scrape failed - beside a healthy `/health` and `LlmEndpointDown` fired for days. - `METRICS=1` is now the default and the seam pins it. -- Deployed 13:16 (`deploy.sh dankclaw --service frank-zealot`) — and the - first boot was **not the shipped payload**: a leftover - `frank-zealot.service.d/ptq1.conf` from the bench window pointed - `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit - booted that arm. `/health` 200, `/v1/models` echoed the alias, Playwright - 33/33 — **nothing in the gate reads the payload.** Drop-in moved to - `~/.AGENTTRASH/`, `daemon-reload`, restart: the shipped trio booted. (The - drop-in was not a rogue — see the PTQ1_0 entry below.) -- Measurement law: the first battery reported 117 and 72 t/s because it - timed natural generations that stopped at 24 tokens — wall-clock aggregate - on an early EOS reads 4× low. `ignore_eos` + the engine's own `timings` - numbers is the only honest way to time this endpoint. -- OpenWebUI `model` rows re-keyed in the live DB. Note for the next edit: - the values live inside the `params`/`meta` JSON columns — the table has no - `context_length`/`description` columns — and there is no `sqlite3` binary - on the host, so `python3` stdlib + `Connection.backup()`. - -## 2026-10-18 — frank-zealot PTQ1_0 arm: measured, approved, and never written down -- **Correction to the entry above and to my own report.** The - `frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished - experiment. The session archive (`01a14c93`) carries Frank's own - instruction at **02:36**: replace frank-zealot with - `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8** mmproj, batch/ubatch - **3072/1536**, "fit as much context as possible in 2 contexts". -- It was the production payload **03:14:39 → 03:38:57** and served real - traffic; recovered from its own `print_timing` lines, PTQ1_0 ran - ≈47.6–49.0 t/s single-stream greedy decode (25–27 t/s per stream with two - streams live) and prefilled deep prompts faster than the PQ2_0 27B record - ever managed. -- Against the PQ2_0 27B record: PTQ1_0 freed the **1,201 MiB** its fetch - script promised and paid **−37 % decode** — the publisher's "dense - ternary unpacking loses on Ampere" warning, now confirmed on GA102. - Frank scratched the whole 27B plan at **03:33** ("scratch the current - frank-zealot plan completely… 9B… enable mtp… if you can possibly fit a - third context, do it"), which is what shipped at 13:16. -- The durable failure is mine: those numbers lived only in `/tmp` and the - journal, never in the repo, so the next agent on the host could not tell - an approved payload from a rogue drop-in and called it a hijack. The - fetch script's header now carries the measurement; - `/home/dank/models/Bonsai2-PTQ1_0/` stays on disk as the 27B rollback and - the drop-in stays trashed. - -## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot - -- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33, and - the drop-in check the zealot incident bought ran FIRST: `DropInPaths` - empty, so the repo env governed. 2026-10-18 — frank-dragoon: the KV pool - doubled to 401,408 positions over the 200,704 ceiling; the measured boot - ledger owns the shape. See [[frank-dragoon]]. -- Warm reuse survived the raise: one **101,987**-token prompt took 81.5 s - cold and 0.3 s on repeats — `HALOGEN_CACHE_INPLACE=1` keeps the KV rows in - the pool, so a warm turn prefills nothing and the doubling evicted nothing - the cache needed. The RAM price is still real: host `MemAvailable` fell - **26 → 22 GiB**, because every pool position is RAM the n-gram table's - page cache loses. -- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the - engine reserves **11.0 GiB** and its preflight bills **11.3**. The measured - figure wins, and every "committed but not yet deployed" clause was retired - with the measurement (AGENTS.md, the env lore block, both index pages, the - iommu page, the runbook). -- PTQ1_0 provenance corrected (see the dedicated entry): the arm that - hijacked the zealot deploy was a **repo-tracked, Frank-ordered, measured** - candidate — `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, - and its header now carries the recovered numbers (≈48 t/s decode, 801 t/s - prefill at 141k) and the verdict: −37 % decode against PQ2_0 for 1,201 - MiB freed, then scratched by Frank's 03:33 order in favour of the 9B. - Nothing is owed on it. -- Census of the halogen HTTP surface while doing it, because the fleet bench - law quoted the wrong engine: frank-dragoon answers `/health`, `/metrics`, - `/v1/models`, `/v1/chat/completions`(POST) and 404s `/ping`, `/props`, - `/completion`, `/tokenize`; the chat response carries **no - `Server-Timing`** header. So the `/completion` `timings` object and the - `ttft;dur=` header are **llama.cpp** instruments — frank-zealot's, not - dragoon's. AGENTS.md's Benchmarking Policy re-keyed to say which engine - owns which instrument. - -## 2026-10-18 — frank-tts phase 1: the cloned-voice endpoint, named and tested (not deployed) -- Voice-over goal, phase 1 (Gitea issue #2). New service **frank-tts** on - Dankclaw: `dankclaw/voiceservice/` (stdlib-only `app.py` + `Containerfile` - + rootless quadlet), `scripts/frank-tts.env`, `tests/frank_tts_unit.bats`. - One OpenAI-shaped `POST /v1/audio/speech` surface; `voice=` is - honoured only when `/voices//reference.wav` exists, otherwise a 400 - listing the voices that are — **no silent fallback voice**; oversized - input is a 414, not a half-narrated sentence; `/health` reports backend + - voice list; no audio ever reaches the log. -- **Naming law applied (Frank's call):** treat it as a first-class model - endpoint → alias-named `frank-tts` (TTS endpoint, so not `frank-asr`), - **port 7772**, joining `7770 frank-dragoon` / `7771 frank-zealot`; the - engine upstream keeps 8880 and is deliberately not a fleet endpoint. The - internal knob names stay `SHIM_*` (they name the façade component, not the - service). -- **Voice-pack law recorded** in the env header: a pack needs - `reference.wav` (24 kHz mono, 18–25 s) **and** the pinned engine's own - speaker artifact — required, beside the WAV, never replacing it — plus - `manifest.json` provenance whose offsets are always past the first 10 s of - the source (the source-material law). Enforcement is split honestly: the - shim gates on `reference.wav` (the only engine-agnostic term); the pack - builder is what will refuse to emit a pack without its artifact. -- Landed as **PR #4** (branch `wt/20261017-181502` → `main`, merged as - `49c4e19`) — the first PR on this repo through the API, which cost three - 422s to get right: the merge endpoint takes **`{"Do":"merge"}`**, `Do` - being a *string* naming the strategy — `DoMerge`/`merge_method` do not - exist on this Gitea, and swagger JSON is disabled on the instance, so the - 422 messages are the schema. Law saved to the `gitea-api-pr-operations` - skill. - -## 2026-10-18 — TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion -- Frank listened to the four audition pairs (`qwentts-*.mp3` against their - `armB-*.mp3` mainline partners) and **passed every clone**. That deleted - the only axis the A/B could not measure, and the question was reframed: - **which engine is better for live output**. On that criterion RTF only has - to stay under 1.0 with margin — the discriminators are time-to-first-audio - and run-to-run spread, because an arm whose RTF can cross 1.0 stalls a - live stream mid-sentence. -- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, - 1.7 B talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on - 8880.** The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` - stays in the repo as the rollback. Three consequences are configuration, - not preference: boot-warm every voice, do not pin a cpuset (threads are - `hardware_concurrency()/2`), register every pack with its WAV (a lone - embedding is rejected). -- Two follow-ups deliberately **not** decided: the **0.6 B talker** (never - auditioned — every clip is 1.7 B) and **ICL vs embedding-only** - conditioning. Embedding-only is the shipped default because it is cheaper - and needs no `ref_text`, which this fleet cannot transcribe honestly; - `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice - opt-in for batch work where total time beats first byte. -- **Addressing law corrected** in [[Local TTS and Voice Cloning]]: the pair - may not meet at a host-loopback publish. netavark DNATs published ports - only and a bridge container cannot reach a host `127.0.0.1` bind — the - lesson that 502'd hermes — so `host.containers.internal:8880`, which - `scripts/frank-tts.env` documents, is correct only for a routable bind. - The deployed pair shares one quadlet network and the façade addresses the - engine by container name. -- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`) is **merged** as - `1eb8b21` (Frank's OK): open-notebook + SurrealDB quadlets on the private - bridge `10.89.4.0/24`, the API published on **5056** because seerr holds - `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT` moved 600 → **660** to sit - one round-trip outside frank-tts's own `SHIM_TIMEOUT`. Merged is not - runnable: nothing listens on 7772 until the voice stack lands, and the - episode destination stays interim pending Gitea #7. - -## 2026-10-18 — open-notebook: the podcast generator's config lands (nothing deployed) - -- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — - notes and sources in, a multi-speaker episode out, script written by - **frank-zealot**, voices rendered by **frank-tts**, no cloud provider - anywhere in the path. Repo-side only: no host touched. Runbook: - [[Open Notebook]]. -- **Two containers, not one.** `lfnovo/open_notebook:v1-latest` plus - `surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` — - frank-dragoon's pinning style, because `:latest` moving underneath a - rootless service is how you get an unbootable pod after an unrelated - `systemctl-daemon-reload`. Both health probes are argument-only - (`curl -fsS …/health`, `/surreal is-ready`), and `is-ready` refuses to - pass without `SURREAL_PASS` in the environment — a credential that must - never ride the command line, so `Exec=` carries flags only. -- **Host 5055 was already spoken for** — seerr has held `127.0.0.1:5055` — - so the API publishes `0.0.0.0:5056:5055`, and because open-notebook's - frontend derives its API URL from the window origin when `API_URL` is - unset, `API_URL=http://192.168.0.143:5056` is set to match: a silent - remap would hang the UI on a port nobody listens on. 8502 publishes on - `0.0.0.0` because this service carries its own password (the - episode-audio route is unauthenticated); **8000 is published nowhere**, - where upstream exposes it on loopback with root credentials, and the seam - forbids it. -- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread - engine per request on a box that also serves two LLMs, while - open-notebook defaults to a 25-way self-DoS; both knobs are pinned to - `1` in `scripts/open-notebook.env`. The TTS clock goes to **660 — one - round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so a shim - give-up reads as the shim's error rather than the worker's; the seam - asserts the direction (`>`) against the frank-tts env instead of - restating a number. -- **What a credential needs in order to exist:** - `OPEN_NOTEBOOK_ENCRYPTION_KEY` must be set before the UI can save *any* - provider credential, and `OPEN_NOTEBOOK_PASSWORD` must stay **absent** to - remain auth-off — the app reads `os.environ.get(…) is not None`, so - `PASSWORD=` boots a login form asking for the empty string. The deployer - seeds `/home/dank/Containers/open-notebook/env` (600) from the host vault - and **refuses to start the pair while either secret is blank**, because a - half-configured open-notebook stores no credentials and fails quietly. -- **Episode destination is interim, by decision.** The destination is - **Gitea issue #7, `human-only`** — agents must not work it. Until Frank - settles it, the app keeps its own named volume and - `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` - (`DATA_FOLDER` is hard-coded `./data`), so finished MP3s are ordinary - files in his home directory instead of volume contents needing - `podman unshare`. One mount line; one deletion undoes it. -- **Two UI buttons will fail, and that is cosmetic:** Test Connection does - `GET /models` and the TTS probe synthesises voice `alloy`; - frank-tts serves `/health` and `/v1/audio/speech` only. Model rows are - created *by name* and the voice field is free text, so `narrator1/2/3` is - typed in and never discovered. Documented, not worked around. -- Three deployer seams cost time and are now law-shaped: a service must be - registered in **five** places — `SERVICE_ORDER`, `DANKCLAW_SERVICES`, the - `deploy-remote.sh` dispatch case, its pre-deploy requirement, and - `deploy.sh`'s dispatch regex, where an omission silently plans an empty - set for `--only`; a `dankclaw//*` case must precede the - `dankclaw/*.container` glob because that `*` crosses `/`; and the - changed-path routing lives *inside* the per-path loop, after the - `scripts/*.env)` case, not at the top of the file. - -## 2026-10-18 — frank-tts-engine: the arm A engine becomes deployable (nothing deployed) -- Goal: turn the engine the day's A/B picked into a **repo-owned, - boot-persistent, self-healing container** — `frank-tts-engine`, - `qwentts.cpp` commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`, resident - `tts-server`, container port 8880, and **not** a fleet endpoint: the - endpoint stays `frank-tts` on 7772. Runbook: [[Frank TTS Engine]]. -- **No `PublishPort` at all, which the merged A/B states as law.** netavark - DNATs published ports only, so a `127.0.0.1:8880` publish is not a path to - a peer container (the trap that 502'd hermes) — while being the one - LAN-reachable surface of an unauthenticated synthesiser. Both voice - containers therefore sit on one committed quadlet network, - `frank-tts-engine.network` at **10.89.5.0/24** (the next free /24 after - open-notebook's 10.89.4.0/24), the façade addresses the engine by - container name, and diagnostics — including the deployer's health gate - (`wait_container_healthy`, there is no URL to dial) — run through - `podman exec`. -- **The boot warmer is a unit hook, not an operator habit.** The registry is - memory-only and the first request for a registered voice decodes the whole - reference, so `ExecStartPost=-/usr/bin/podman exec frank-tts-engine - /app/warm-voices.sh` with `TimeoutStartSec=900`; the `-` prefix is what - stops a failed warm from crash-looping the endpoint under - `Restart=always`, and it logs `cold=` and `warm=` per voice on every boot. -- **Thread law recorded before anyone tunes it away:** no `--threads` - exists; `src/backend.h` uses `hardware_concurrency()/2`, so a cpuset pin - silently changes the thread count. Nothing pins this container. -- Image builds from the pinned commit with the **proven** line, verbatim — - `GGML_BLAS=OFF` forced, every GPU backend off, no `-DGGML_NATIVE` because - ggml's default is `-march=native` for the building host: the image is - built where it runs. Weights fetched by - `scripts/download-frank-tts-engine-qwentts.sh` with byte and LFS-oid - checks; the deployer gates on `--verify-only` rather than pulling 1.4 GiB - inside a deploy. - -## 2026-10-18 — frank-tts: the 7772 endpoint gets its deploy wiring, and voice naming is written down - -- Stage 2 of the voice stack: the **façade** in front of the engine is now a - deployable service too. Runbook: [[Frank TTS]]. -- **Two attached networks, in a meaningful order.** `dankclaw_default` - first — it owns the default route and is where consumers live - (open-notebook dials `http://frank-tts:7772/v1` by container name) — then - the voice pair's private bridge. `scripts/frank-tts.env`'s `TTS_UPSTREAM` - moves from `host.containers.internal:8880` to - `http://frank-tts-engine:8880`, the actual fix for a shape that could - never have worked: a bridge container cannot reach a host-loopback bind, - and the engine publishes nothing. -- **`Wants=` + `After=` on the engine, never `Requires=`:** the façade still - carries the `llama-tts-cli` rollback backend, which needs no upstream, so - an engine that will not boot must delay this endpoint, not take it down. -- The façade handler installs the **engine's** quadlet pair as well — not - tidiness: with a referenced `Network=` file absent the v1 Quadlet - generator produces no container unit at all — and it `mkdir -p - /home/dank/voices` as the deploy user, because rootless podman - auto-creates a missing bind source **root-owned** and the endpoint then - reports "no voices" instead of an error. -- **A voice is a lowercase directory name.** `^[a-z0-9][a-z0-9._-]*$`, - ≤ 64 chars, no slash/`..`/space. The shape is the intersection of two - rules that do not agree: `app.py` would accept mixed case, `tts-server` - lowercases names — so a pack named `Narrator1` resolves through the façade - and not through the engine, and the boot warmer refuses such a directory - loudly. `narrator1/2/3` are load-bearing for open-notebook. -- **No Prometheus job, on purpose.** The façade exports no `/metrics`, - Prometheus 3 refuses HTML/JSON content types (the `comfyui` and `services` - probes are documented down for exactly that), and `LlmEndpointDown` is - scoped `up{job=~"frank-.*"}` — so a `/health` scrape would page on Signal - every few minutes for a healthy endpoint. That is the frank-archon - never-green lesson pointed the other way, written in the page instead of - quietly ignored. - -## 2026-10-18 — voice-pack builder: the 10-second source-material gate becomes code - -- Third slice of the voice stack: `scripts/build-voice-pack.sh` turns a - source recording plus a crop offset into a deployable pack — - `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the engine's own - `reference.spk` + `reference.rvq` made from it, and `manifest.json` - provenance. Tracked tunables in `scripts/voice-packs.env`; new runbook - [[Voice Packs]]. -- **The gate is the product.** The project-wide invariant — no voice - material from the first 10 s of a sample file — had only ever been a - sentence in `scripts/frank-tts.env`. An offset inside it now exits 1 - before a directory is created, and the same assertion runs again against - the manifest that was just written, next to the recorded sha256, so a pack - cannot carry a forbidden crop even if the env file is edited into - nonsense. `10` stays a constant in the builder; the env file carries - offsets, never the law. -- **The extractor's contract was read, not guessed:** `qwen-codec` in the - pinned qwentts.cpp commit has **no `-o`** and names both artifacts next to - the input with the extension swapped. So the WAV is staged under its final - stem and `reference.wav` — the file the façade gates on — is moved into - place **last**, after the artifacts and the manifest: a failed extractor - leaves a directory that is not a voice. `--force` retires an existing pack - whole to `.AGENTTRASH/` instead of touching bytes; nothing in this script - deletes. -- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B - per 12.5 Hz frame and `enc_dim 2048 × f32` = 8,192 B. That makes the - **effective crop window 19.84–25 s**, narrower than the WAV law's 18–25 s, - and a too-short crop is refused before the extractor runs. An engine bump - that changes either number fails the build by design. - -## 2026-10-18 — voice stack audit: three faults that no gate could see -- The voice stack had never been run end to end, so the whole path was - audited as static code against the tools it calls, and three faults came - out of it. All three are the same species: a step that **reports success - while doing nothing**. -- **The build would not have happened.** `Containerfile.engine` created its - unprivileged user with `adduser`; the pinned ubuntu:24.04 base has no - `adduser` at all — proven in the base on Dankclaw, not argued from memory. - Replacement: `useradd --system --uid 10000 --home-dir /app --shell - /usr/sbin/nologin tts`; its `SYS_UID_MAX` warning is cosmetic and is the - price of matching the façade's uid. -- **The warm pass would not have warmed.** The unit hands its env file to - podman via `--env-file`, which performs no shell parsing and takes the - literal remainder of the line, while `source` strips one quoting layer. - `WARM_TEXT` therefore reached the container with its quotes, the JSON was - invalid, and every voice would log "registered, but the warm synthesis - failed" beside an `active` unit. Fixed in the hook; the bug survived the - first draft precisely because every existing test used `source`. -- **A rebuilt image would not have been rebuilt.** `build_compose_image` - skips `podman build` on a matching `cfhash` label, and that hash covered - the `Containerfile` alone — so editing `app.py` or `tts-engine-warm.sh` - (both `COPY`'d) skipped the build while the deploy reported success and - `/health` answered with old code. Each case arm now hashes the files its - `Containerfile` copies. -- Docs carry the lessons where an operator will hit them: a **Locally built - images** section in [[deployment]], two new troubleshooting lines in - [[Frank TTS Engine]] (all-voices-cold = quoted `WARM_TEXT`; façade 502 - beside 200 = model loaded ≠ voices registered), and `Voice Packs.md` - step 4 corrected — the warm hook re-registers **every** pack on every run, - because the engine replaces a name it already holds, so a restart is what - an engine bump needs and not what a rebuilt pack needs. - -## 2026-10-18 — frank-tts-engine deployed: it builds, it serves, and the warm hook died on the kernel's argument cap -- Merged as **#14** (`747c665`) and deployed `--service frank-tts-engine`; - the image built clean — the first real proof of the `useradd` fix — and the - unit was `healthy` after 31 s. -- Then the deploy reported `0 voice(s) warm`: - `/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long`. The - hook handed the registration body to curl as ONE argument; the kernel caps - a single argument at 131,072 bytes (`MAX_ARG_STRLEN`) and a shipped ~955 KB - `reference.wav` base64's past that, so curl never executed and the hook - logged that as "HTTP no response". Fourth silent fault of the voice stack, - same species as the other three: `/health` stayed green throughout. Fix: - the body goes to a `mktemp` file and curl gets `@file`; the regression test - is the honest shape — real listener, real `curl`, a 300 KB fixture pack — - so it fails with the same kernel error the host produced. Every shipped - pack sits well over the ~96 KB cliff, so no small fixture could ever have - seen this. -- **A trial law did not survive deployment.** Against an empty registry the - first synthesis of a freshly registered voice cost ~1.9 s, the next ~1.2 s. - The trial's 6.2–8.0 s cold / 0.5 ms restore ladder, which the warmer's own - header repeated, is retired: warming buys registration — mandatory, because - the registry is memory-only — plus about **0.6 s**. Corrected in - [[Frank TTS Engine]] law 1 and its troubleshooting table, and in - [[Local TTS and Voice Cloning]] and [[Voice Packs]]. -- Sweep afterwards: the retired ladder still lived in six places the first - pass missed — both quadlet files, `scripts/frank-tts-engine.env`, - `scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row, the - research brief's consequences paragraph, and one test comment. All - re-keyed to the measured claim. The trial entry earlier in this log is - left as written — a dated worklog entry is a record — and this line is the - correction that points at it. - -## 2026-10-18 — 7772 is live and the harness speaks through it -- Merged **#15** (`0315a62`, the warmer's argument-cap fix) and **#16** - (`d2ce669`, the comment sweep), then deployed both voice services in order - (`--service frank-tts-engine`, then `--service frank-tts`); Playwright gate - 33/33 twice. Deployed-state proof from a LAN client: `/health` → - `omni-http` with all three voices, and the same sentence in `narrator1` vs - `narrator2` came back with different sha256 and different RMS — the name - actually selects a voice. -- **The harness `speech` role moved off Kokoro**: `config.yml` - `modelRoles.speech: frank-tts (dankclaw)/frank-tts`. No voice key moves - with it. -- Two harness laws learned by breaking it first (both now on [[Frank TTS]]): - an explicit speech role has **no fallback candidate**, so a dead endpoint - errors instead of degrading; and `models.yml` loads at process start while - `config.yml` is read live, so a session older than the `frank-tts` row - fails exactly like a dead endpoint. -- Corrected the same evening, by breaking it in a restarted session: a - harness call that names no voice reaches 7772 with **no `voice` key at - all** — `400 unknown voice None; available voices: …` — because no setting - feeds one. The endpoint therefore grew `DEFAULT_VOICE` (its own PR): a - voice-less request speaks the declared pack, an unknown name still 400s - with the list. `speech.voice: narrator1` — briefly recorded as wiring — - went back to `am_fenrir`: that key is the vocalizer's Kokoro enum, so a - pack name there breaks speak-aloud and does nothing for synthesis. -- Status sweep: six pages still said the endpoint did not exist — both vault - descriptions, the two `index.md` bullets, the Open Notebook blocker (the - voice leg works; open-notebook itself is what is missing), the engine - page's deploy order (now a deploy record), and the homepage card, which - lost `wip: down` so the admin panel probes 7772 for real. - -## 2026-10-18 — the default voice shipped: a harness voiceover needs no voice -- Merged **#17** (`b590695`) and **#18** (`7889a68`) and deployed - `--service frank-tts --service homepage` in one dispatch, Playwright gate - 33/33. The call that had failed is the one that proves the fix: same - device, **no** `voice_id` — `Saved 430124 bytes (…, voice=default, - codec=wav)` — and the façade journal says what actually spoke, - `speech voice=narrator1 (default)`. The harness prints `voice=default` - because it never had a name to print; the voice that answered is the - operator's `DEFAULT_VOICE` line in `scripts/frank-tts.env`, and the journal - keeps them distinguishable forever after. -- The page gained a **Default voice** row in its contract table and - [[#Naming a voice]] now states that a call naming nothing is the harness's - normal shape, not a client bug — the misconception that produced the 400 in - the first place. - -## 2026-10-18 — OpenWebUI "voice narrator1 not found": the rejector is in the browser -- Frank's toast — `Error: Voice "narrator1" not found. Should be one of: - af_heart, …, bm_fable` — looked like frank-tts refusing the name. It is - not: that 29-name enum is the voice pack of **Kokoro.js running in the - page** — OpenWebUI 0.11.4 ships that implementation in its own front end. -- Instruments that separated the two paths, in order: the admin config DB is - correct (`audio.tts.engine=openai`, base `http://host.containers.internal:7772/v1`, - `voice=narrator1`); from inside the container the exact OpenWebUI body - returns **200 `audio/mpeg`**; and `podman logs open-webui` holds **zero** - `audio/speech` requests for the failing click. A server-path failure leaves - a log line and a frank-tts error — a `browser-kokoro` failure leaves - neither — and no container env or `user.settings` row carries an override, - so the engine choice is per-browser storage, not fleet state. -- Fix is a UI click, not a deploy: Settings → Audio → TTS Engine → - **Default** hands synthesis back to frank-tts and the cloned voices; - choosing `af_heart` keeps in-browser Kokoro, which cannot clone (fixed - pack). Recorded on [[OpenWebUI]] under *Text-to-speech (frank-tts)*. -- Also in the window: **PR #22 merged** (`0c7fe8b`, `dank/command-center`) so - `agent/extensions/dankvoice.ts` landed on the auto-load path — proven in a - plain `omp` session with no flags: `/say merged build check` → WAV to the - player, temp dir cleaned. Sessions open before the merge need a restart to - pick the command up. - -## 2026-10-18 — frank-zealot: 2 × 262,144 over unquantized f16 KV -- 2026-10-18 — frank-zealot: shape cut to 2 × 262,144 over unquantized f16 KV - (the 786,432 pool cannot fit exact KV on the 3090; concurrency 3 → 2, and - MTP is now doubly out — off by the 13:16 measurement, and exact KV bills a - draft-context f16 pool over the whole context). See [[frank-zealot]]. -- Arithmetic correction to the repo's long-standing figure: "the f16 pair - would need **26,112 MiB**" doubled the q8_0 number *including* its 1/32 - scale overhead — q8_0 is 17,408 B/token (1.0625 B/element), so f16/q8_0 is - **1.882×**, not 2×, and exact f16 over 786,432 is **24,576 MiB**. The - conclusion stands (three f16 windows cannot boot this card); the number was - wrong. -- New gate in `tests/frank_zealot_unit.bats`: dtype × pool plus the measured - non-KV terms must fit the driver-addressable card with **≥ 1 GiB** of slack - — and the same gate proves it refuses the impossible arm (f16 at 786,432), - not merely passes the shipped one.