docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
This commit is contained in:
Frank
2026-10-19 05:00:43 -07:00
parent 3880503057
commit a5fe045b58
122 changed files with 2412 additions and 2037 deletions
@@ -1,7 +1,7 @@
---
type: reference
title: Agent Docs and Edits
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
tags: [agents, docs, markdown, hashline, edit-recovery]
timestamp: 2026-10-18T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: Command Center Dotfiles
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
tags: [dotfiles, omp, deploy, rsync, blankpad]
timestamp: 2026-10-05T00:00:00-07:00
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: OMP Harness
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
tags: [omp, harness, compaction, models-yml, context, samplers]
timestamp: 2026-10-18T00:00:00Z
---
+3 -14
View File
@@ -1,23 +1,12 @@
---
type: index
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
tags: [agent, index]
timestamp: 2026-10-05T00:00:00-07:00
---
# Agent
Pages about the agent's own tooling and context — how omp is configured, how
docs get edited safely, and how the agent's home is deployed.
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
`compat.extraBody`, preamble and skill-library audits
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
hashline edit recovery
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
transport, override packages
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
## Related
- [[Testing Policy]] — the seams these pages' checks ride
- [LLM](../LLM/index.md) — the fleet these agents operate
@@ -1,8 +1,7 @@
---
type: reference
title: Container Network Namespaces
description: Which local services are addressable from which Podman network namespace on
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
tags:
- networking
- podman
@@ -1,7 +1,7 @@
---
type: reference
title: Deploy State Triage
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
tags: [deployment, deploy-state, systemd, podman, triage]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,5 +1,5 @@
---
type: guide
type: howto
title: Deployment Flow
description: Service-scoped Git deployment for the two homelab hosts.
tags:
@@ -1,6 +1,6 @@
---
type: reference
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
tags:
- architecture
- e2e
@@ -1,7 +1,7 @@
---
type: reference
title: Quadlet Lifecycle
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Stale Signal Audit
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
tags: [testing, observability, audit, signals, gates]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: howto
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
tags:
- dankclaw
- comfyui
@@ -1,6 +1,6 @@
---
type: reference
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
tags:
- dankclaw
- tts
+4 -1
View File
@@ -1,11 +1,13 @@
---
type: reference
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
tags:
- endpoint
- dankclaw
- tts
- voice
- quadlet
timestamp: 2026-10-19
---
# Frank TTS
@@ -202,3 +204,4 @@ harness bundle and measured the day of the flip:
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
---
+1 -1
View File
@@ -1,5 +1,5 @@
---
type: configuration
type: reference
title: Gitea
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
tags:
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Hermes-WebUI (Hermex server)
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
tags:
- hermes
- hermes-webui
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Marreta on Dankclaw
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
tags:
- dankclaw
- marreta
@@ -1,6 +1,6 @@
---
type: reference
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
tags:
- dankclaw
- open-notebook
+3 -3
View File
@@ -1,6 +1,6 @@
---
type: reference
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
tags:
- dankclaw
- seerr
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
authentik's own user UI instead of Seerr. Both backfilled live: the three
standard flows copied from a known-good provider, plus an application
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
provisioning rules this incident established.
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
two provisioning rules this incident established.
## *arr wiring
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Dankclaw Swap Configuration
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
tags:
- dankclaw
- host
@@ -1,6 +1,6 @@
---
type: reference
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
tags:
- dankclaw
- frank-tts
+13 -6
View File
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
## Security Advisories
| Version | CVE | Issue |
|---------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
| Version | Advisory | CVE | Issue |
|---------|----------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
## Gotchas
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
- [[Dankserver Stack]]
- [[Caddy]] — TLS termination and authentication in front
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
## Citations
- GitHub Security Advisories, `9001/copyparty` —
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
each row's patched version read from `first_patched_version` in the
`https://api.github.com/advisories/<GHSA-id>` response.
@@ -1,6 +1,6 @@
---
type: reference
description: "Dankserver (192.168.0.235) runs the media and library stack."
description: "Service inventory and routing overview for the media and library host."
tags:
- dankserver
timestamp: 2026-09-03
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Jellyfin
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,6 +1,6 @@
---
type: reference
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
tags:
- dankserver
- romm
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: MariaDB (RomM Backend)
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
tags: [mariadb, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Redis (RomM Cache / Queue)
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
tags: [redis, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: Noctalia and Niri
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: "*arr Stack"
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
timestamp: 2026-09-04T00:00:00Z
---
+7 -7
View File
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
## *arr Stack
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
- [[sonarr]] — TV show management
- [[radarr]] — movie management
- [[lidarr]] — music management
- [[readarr]] — ebook/audiobook management
- [[prowlarr]] — indexer management
- [[qbittorrent]] — torrent download client
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
@@ -1,7 +1,7 @@
---
type: reference
title: Prowlarr
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
tags: [prowlarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: qBittorrent
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
timestamp: 2026-09-03T00:00:00Z
---
+17 -9
View File
@@ -1,18 +1,19 @@
---
type: reference
title: Readarr
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
tags: [readarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Readarr
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
> metadata became unusable and the community migration to Open Library stalled.
> There is no upstream development. Alternatives: the Bookshelf community fork,
> or the Calibre content server. This instance still runs on a pinned image
> tag; see below.
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
> Readarr: its metadata became unusable and the community migration to Open
> Library stalled. There is no upstream development — the GitHub repository is
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
> community fork, or the Calibre content server. This instance still runs on a
> pinned image tag; see below.
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
@@ -69,9 +70,9 @@ Restart=always
## Gotchas
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
`:latest` does not exist for this image on LSIO.
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
because `:latest` does not exist for this image on LSIO.
- Calibre integration is folder-based, not API-based. Readarr drops files into
the books root which [[Calibre-Web]] serves.
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
@@ -81,3 +82,10 @@ Restart=always
- [[ARR-Stack]] — full *arr stack documentation
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
## Citations
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
@@ -1,6 +1,6 @@
---
type: reference
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
tags:
- infrastructure
timestamp: 2026-09-17
@@ -0,0 +1,230 @@
---
type: reference
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
tags:
- infrastructure
- authentik
timestamp: 2026-10-19
---
# Authentik Outpost Reference
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
self-serve user-settings flow, and the live database shape. What the service is,
how it is deployed, wired and operated is [[Authentik]]; the three core
containers are [[Authentik Processes]].
## Live proxy-provider inventory
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`,
`default-provider-authorization-implicit-consent`,
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
## Provisioning rules for proxy providers
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`,
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
## LDAP provider objects (provisioned via `ak shell` ORM)
The LDAP outpost's own deployment — unit, image digest, published ports, token
env var — is in [[Authentik]]. The objects it serves:
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** An earlier
revision asserted that a bind token with `intent=app_password` is rejected by
the flow's password stage with `Invalid password` and only
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
wrong or the current Jellyfin bind path is not that token. Nothing observable
settles it today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
*(unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Live database shape
For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Processes]] — the postgres, server and worker containers
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
@@ -0,0 +1,76 @@
---
type: reference
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-10-19
---
# Authentik Processes
The three core containers of the [[Authentik]] stack on Dankclaw. The two
outpost units are not covered here — they live on [[Authentik]] and
[[Authentik Outpost Reference]]. Server and worker run the same image; the
`command` selects the role (`server` / `worker`). Data persists under
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
standardization; the retired stubs this page replaces recorded the pre-move
paths `/home/dank/authentik/data` (server and worker) and
`/home/dank/authentik/postgres` (database).
## authentik-postgres
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
PostgreSQL in 2025.10 (Redis removed).
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
volumes)
- **Health:** `pg_isready -d authentik -U authentik`
- **Retired wording:** the stub called this a "Compose service … in
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
- The side-by-side with `gitea-postgres` (network, published ports, secret
source, boot persistence) is owned by [[Postgres]].
## authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
It also serves the login page at `https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
(`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
live here
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
worker)
- **Health:** recorded by the retired stub as
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
owns the live probe.
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
integration.
## authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
(`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
the data dir)
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
server)
- **No port** — the worker has no HTTP endpoint.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Outpost Reference]] — provider, flow and token configuration
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
+23 -212
View File
@@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
they handle their own session. Grafana is the exception that is both: it is a
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
`dankclaw/Caddyfile:374-377`; live provider table below).
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
@@ -35,6 +35,13 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
pre-quadlet era, retired.
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
**Split off from this page:** [[Authentik Processes]] owns the per-container
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
the live proxy-provider inventory, the proxy-provider provisioning rules, the
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
design, and the live database shape.
## Why Authentik
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
@@ -100,23 +107,10 @@ contract:
The chain terminates there — nothing dials back into Authentik from
`:8017`.
## The three compose services
## The stack's processes
| Service | Role | Image |
|---|---|---|
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
moved to PostgreSQL in 2025.10 (Redis removed).
- **Server** and **worker** run the same image; the `command` selects the role
(`server` / `worker`).
- The worker runs as `0:0` (rootless: container root maps to the host user that
owns the data dir).
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
in the 2026-09-01 quadlet standardization).
Per-container facts — unit path, image and role, port, run-as, data directory,
health probe — are owned by [[Authentik Processes]].
## Caddy integration
@@ -142,104 +136,9 @@ routing table, and the Authentik proxy-provider list must be the same set of
hosts. A guarded host with no provider 404s at login time; a provider with no
DNS record breaks off-LAN clients.
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`,
`default-provider-authorization-implicit-consent`,
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`,
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
the two provisioning rules the outpost enforces, and this fork's OAuth2
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
## LDAP outpost (Jellyfin auth, live 2026-09-01)
@@ -267,44 +166,9 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
from the host `dankclaw/.env` (git-ignored).
### Authentik-side objects (provisioned via `ak shell` ORM)
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** This page
long asserted that a bind token with `intent=app_password` is rejected by the
flow's password stage with `Invalid password` and only `intent=verification`
passes. The live token for `svc-jellyfin-ldap` is `app_password`
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
current Jellyfin bind path is not that token. Nothing observable settles it
today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
*(unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
The objects behind it — LDAP provider, bind DN, service account and bind token,
RBAC grant, groups, application slug — and the hard-won bind rules are in
[[Authentik Outpost Reference]].
## Configuration (secrets)
@@ -340,7 +204,8 @@ every outpost session (see the ORM playbook).
Applications** → add an application + a **Proxy** provider whose
`external_host` is exactly the browser-facing origin
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
flows set (see the provisioning pitfall above). Attach the provider to the
flows set (see the provisioning rules in
[[Authentik Outpost Reference]]). Attach the provider to the
embedded outpost.
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
the app with the correct redirect URI, and paste the `client_id` /
@@ -373,47 +238,6 @@ every outpost session (see the ORM playbook).
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
Authentik migrates the schema on startup.
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Outpost claim cache (`proxysession`) — 403 after a group add
The embedded outpost does not re-evaluate group membership per request. Each
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
(the inventory table carries both bare origins and trailing slashes;
routing matches on hostname, verified 2026-09-02).
`internal_host = http://caddy:8017`, all three flows set, and an
Application attached — the two provisioning rules above.
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
2. **Route flip in ONE commit/deploy**: the vhost becomes
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
inside the `http://:8017` listener add `@<svc> host <vhost>` +
@@ -521,22 +345,9 @@ EOF
is no `authentik.policies.group` module in this build.
*(unverified 2026-10-18: these module paths were not re-executed — importing
outside a configured Django shell fails with `ImproperlyConfigured`)*
- For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
- The live database's table and column names (the 2026.x renames, the MTI parent
keys, the read-only `psql` audit recipe) are reference lookup:
[[Authentik Outpost Reference]].
## User provisioning playbook (user-level account)
@@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
Basic client auth, form-encoded token body, and the fork's userinfo-claims
bug that leaves identity at the `sub` hash only) is in
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
@@ -1,7 +1,7 @@
---
type: reference
title: Homepage Status Probes
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
tags: [homepage, status, probes, e2e, homepage-status]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: reference
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
tags:
- infrastructure
- homepage
@@ -1,7 +1,7 @@
---
type: reference
title: Observability Provisioning
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
tags: [observability, prometheus, grafana, provisioning, deployment]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: "Observability on Dankclaw"
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
timestamp: 2026-09-11
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
tags:
- infrastructure
- prometheus
@@ -1,24 +0,0 @@
---
type: reference
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-08-28
---
# authentik-postgres
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
OIDC provider. It is the only stateful store in the stack (users, flows,
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
now live here.
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
- **Health:** `pg_isready -d authentik -U authentik`
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
(`pg_dump`), and failure modes.
@@ -1,24 +0,0 @@
---
type: reference
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
This is the service the Caddy `forward_auth` vhosts proxy to
(`authentik-server:9000`). It also serves the login page at
`https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
- **Port:** `9000` (published; the outpost + admin + login all live here)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
- **Health:** `http://localhost:9000/-/health/`
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
forward_auth pattern, first-time setup, and failure modes.
@@ -1,23 +0,0 @@
---
type: reference
description: "The Authentik worker — background tasks: event processing, provider"
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal. It runs the same
image as the server; the `command: worker` selects the role.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
- **No port** — the worker has no HTTP endpoint.
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
modes.
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Dankserver System Specs
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
tags:
- dankserver
- hardware
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Dual R9700 Cooperation
description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
tags:
- frank-medium
- rocm
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: FastFlowLM NPU Inference
description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
tags:
- fastflowlm
- npu
@@ -1,7 +1,7 @@
---
type: reference
title: FreeToken Builds
description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
tags: [freetoken, build, dankclaw, vllm, mtp]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: GGUF Compatibility and Runtime Lock-in
description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
tags: [gguf, llama.cpp, forks, quantization, speculative]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Heterogeneous Split Triage
description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: LLM Bench Windows and Measurement
description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: LLM Performance Diagnosis
description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Llama Cpp Multi-Device Splits
description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Llama Cpp Serving Sizing
description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Model Swap and Rollback
description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: PCIe Link and Atomics
description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,11 +1,11 @@
---
type: reference
title: Ethernet Bandwidth Math
description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-17
timestamp: 2026-10-19
---
# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
@@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2 [fp16 = 2 bytes]
- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
**Worked examples (1 boundary = 2 PP stages):**
```
1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
```
**Example — same model, batch=64:**
```
1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
```
**Example — 36B model, 2 PP stages, batch=1:**
```
1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
27B, batch=1: 1 boundary × 4096 × 2 bytes = 8,192 bytes/tok = 8 KB
27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288 = 512 KB/tok
36B, batch=1: 1 boundary × 6144 × 2 bytes = 12,288 bytes/tok = 12 KB
```
### 1.2 Tensor Parallelism (TP) — for comparison
@@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2 [L all-reduces total
| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
### 2.3 Feasibility matrix
### 2.3 Feasibility
Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
**27B model — PP (1 boundary), batch=1:**
| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
|---|---|---|---|
| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
**27B model — PP (1 boundary), batch=16:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
**27B model — PP (1 boundary), batch=64:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
**27B model — TP (32 layers), batch=1:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
**27B model — TP (32 layers), batch=16:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
**27B model — TP (32 layers), batch=64:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
feasible on every link class — even the heaviest, 27B at batch=64 and 100
tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
except batch=64:
**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
@@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
**Petals [1]:**
- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)
**llama.cpp RPC [2]:**
- Model weights and KV cache distributed across nodes proportionally to memory
@@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute
## Appendix: Full bandwidth formulae
### Pipeline Parallelism (generation, batch=1)
### Pipeline and Tensor Parallelism (generation)
`N` is the batch size; `N = 1` is the batch=1 form.
```
bytes_per_boundary_per_tok = hidden_size × 2 [fp16]
total_bytes_per_tok = num_boundaries × hidden_size × 2
bytes_per_boundary_per_tok = hidden_size × N × 2 [fp16, PP boundary]
bytes_per_layer_allreduce = hidden_size × N × 2 [TP, one all-reduce per layer]
total_bytes_per_tok = num_boundaries × hidden_size × N × 2 [PP]
total_bytes_per_tok = num_layers × hidden_size × N × 2 [TP]
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
```
### Pipeline Parallelism (generation, batch=N)
```
bytes_per_boundary_per_tok = hidden_size × N × 2
total_bytes_per_tok = num_boundaries × hidden_size × N × 2
```
### Tensor Parallelism (generation, batch=1)
```
bytes_per_layer_allreduce = hidden_size × 2
total_bytes_per_tok = num_layers × hidden_size × 2
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
```
### Tensor Parallelism (generation, batch=N)
```
total_bytes_per_tok = num_layers × hidden_size × N × 2
```
### With 8-bit quantization (like Petals)
```
# Simply divide all fp16 byte counts by 2
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
```
@@ -1,7 +1,7 @@
---
type: reference
title: Expected Performance
description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
tags:
- pipeline-parallelism
- performance
@@ -13,7 +13,7 @@ timestamp: 2026-08-14
This page estimates the speed of a 25-40B model split across two machines
over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
and the measured numbers in [[Mixed-Vendor Experiments]].
and the mixed-vendor measurements in [[Multi-Node Runtimes]].
## The Bandwidth Ceiling Is Not the Limit
@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
## See Also
- [[Ethernet Bandwidth Math]]
- [[Mixed-Vendor Experiments]]
- [[Multi-Node Runtimes]]
- [[Sample Script Set]]
@@ -1,9 +0,0 @@
---
type: reference
title: "Mixed-Vendor Experiments"
description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -0,0 +1,257 @@
---
type: reference
title: Multi-Node Runtimes
description: "How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts."
tags:
- pipeline-parallelism
- llamacpp
- vllm
- speculative
timestamp: 2026-10-19
---
# Multi-Node Runtimes
Two runtimes can put one model on two machines: llama.cpp over the ggml-rpc
backend, and vLLM with pipeline parallelism. This page owns the cross-runtime
facts — how each is built and launched, what each can and cannot be combined
with, and the mixed-vendor verdicts. Facts owned by a sibling page are linked,
not restated. Bytes-per-token arithmetic and link budgets belong to
[[Ethernet Bandwidth Math]], throughput estimates to [[Expected Performance]],
the real deployment and its verified gotchas to [[Qwen3.8 PP Deployment]], the
educational scripts to [[Sample Script Set]], drafter mechanisms to
[[DFlash DSpark Speculative Decoding]], single-host multi-GPU placement to
[[Llama Cpp Multi-Device Splits]].
## What each runtime is
Both distribute by **layers**, not inside layers.
| | llama.cpp over ggml-rpc | vLLM pipeline parallelism |
| --- | --- | --- |
| unit of distribution | a contiguous slice of layers (`--split-mode layer`, the default) | a contiguous slice of layers (`--pipeline-parallel-size`) |
| process model | one `llama-server`/`llama-cli` on the head; a `ggml-rpc-server` per worker | one `vllm serve` per node, one engine across them |
| what crosses the wire | whatever graph work the head offloads to the remote devices | the hidden state at each stage boundary, point to point |
| KV cache | distributed with the weights; the KV of a layer lives with the owner of that layer | each stage holds the KV of the layers it owns |
| speculation together with the split | yes in llama.cpp (see below) | no, per vLLM's own docs |
Upstream's own summary is why layer-wise is the ethernet choice: pipeline
parallel "minimizes data transfers between GPUs but requires many tokens to
scale well", while tensor parallel "does multiple cross-GPU reductions per
layer … much more bottlenecked by the GPU interconnect speed".
Do not carry the tensor path across machines: `--split-mode tensor` is
experimental, needs flash attention, rejects quantized KV, and is not
implemented for the MoE and hybrid architectures the guide lists — and
per-tensor placement across vendors is not expressible (§1 of [[Llama Cpp Multi-Device Splits]]).
## llama.cpp over ggml-rpc
### Build and launch
Both sides take `-DGGML_RPC=ON` next to that host's own backend:
```bash
cmake .. -DGGML_CUDA=ON -DGGML_RPC=ON # a CUDA worker; -DGGML_HIP=ON for an AMD one
cmake --build . --config Release
```
Worker side — `ggml-rpc-server` exposes ggml devices on the remote host:
```bash
bin/ggml-rpc-server # endpoint 127.0.0.1:50052 by default
bin/ggml-rpc-server --device CUDA0 -c # or CUDA_VISIBLE_DEVICES=0; -c = local cache
```
- It exposes every accelerator it can see; a host with no accelerator exposes a
single `CPU` device. `--device` and `CUDA_VISIBLE_DEVICES` are equivalent
ways to narrow that set.
- `-c` keeps large tensors in `$HOME/.cache/llama.cpp/rpc` (redirect with
`LLAMA_CACHE`) so a big model does not re-stream per load; that cache
filling the worker disk is a recorded fleet incident ([[Qwen3.8 PP Deployment]]).
Head side — build the local backends plus `-DGGML_RPC=ON`, then name the
workers:
```bash
llama-server -m model.gguf --rpc 192.168.88.10:50052,192.168.88.11:50052 -ngl 99
```
`--rpc` is a comma-separated list of `host:port`, one entry per
`ggml-rpc-server`; the head's own devices stay in the pool next to the remote
ones. Launch scripts for this shape: [[Sample Script Set]].
### Split, context, KV
- Default placement: weights **and** KV across all devices, local and remote,
"in proportion to each device's available memory". Override with
`--tensor-split`, whose proportions follow the order in `--device`. Why the
fleet always pins it (a Strix reports a huge GTT window, so the automatic
split lands on the slow device) is [[Qwen3.8 PP Deployment]]'s.
- `-sm layer` (the default) is pipeline parallelism: each device holds a
contiguous slice of layers and the KV cache of a layer lives on the device
that owns that layer.
- KV types are the usual `-ctk` / `-ctv`, `f16` by default. `--fit` is on by
default and sizes unset arguments to device memory; turn it off to hold a
context you set on purpose. Context is one global `--ctx-size` — the RPC
README documents no per-node context or per-node KV knob.
- Upstream's out-of-memory ladder (stated for tensor mode, useful order
anyway): lower `--ctx-size`, then `--parallel`, then `--n-gpu-layers`.
- Transport is TCP; RDMA is negotiated at the handshake with no flag changes
and falls back to TCP unless both peers can use it (Linux RoCEv2 via
`libibverbs`, macOS RDMA over Thunderbolt 5 via `librdma`). It is point to
point — name the peer's RDMA-capable address in `--rpc` or the session stays
on TCP. `GGML_RPC_NO_RDMA=1` forces TCP; `GGML_RPC_DEBUG` traces the backend.
Upstream calls the backend a proof of concept, "fragile and insecure" (see the
Security section). Two limits shape the design: decode is bound by the per-step
round trip, not by the link's bytes ([[Ethernet Bandwidth Math]]), and every
node must run the identical build or the handshake hangs ([[Qwen3.8 PP Deployment]]).
## vLLM pipeline parallelism
> **Provenance.** Flags and mechanics below were recorded 2026-08-14 and are
> not re-checked upstream in this pass (see "Unverified in this pass"); the
> incompatibility lines, the method catalog and the issue history are re-verified.
- `--pipeline-parallel-size` divides the layers into contiguous stages;
`--tensor-parallel-size` still works inside a node, so the canonical two-node
shape is tp = GPUs per node, pp = nodes.
- Two backends: Ray (`--distributed-executor-backend ray`, the worker joining
a `ray start --head`) or multiprocessing (`--distributed-executor-backend mp`
with `--nnodes`, `--node-rank`, `--master-addr`, `--headless` on the
follower). `mp` avoids the Ray dependency; Ray brings the cluster manager.
- What the interconnect must carry: one activation tensor per stage boundary
per forward pass, point to point — not a per-layer reduction. That is the
whole cross-node traffic story; the byte arithmetic and link budgets are
[[Ethernet Bandwidth Math]]. Pin the NIC vLLM talks on
(`NCCL_SOCKET_IFNAME`) rather than letting it guess an interface.
- The PP bubble: with N stages and one sequence in flight, one stage works
while N-1 idle, so pp=2 costs half the machine at batch one. Batching is the
cure — pipeline parallelism "requires many tokens to scale well" — and the
microbatch formula is [[Ethernet Bandwidth Math]] §5.3.
- KV and context: each stage keeps the KV of its own layers, so the pool grows
with the stage count, and `--max-model-len` is not reduced by the split
(recorded 2026-08-14, unverified here).
- What it cannot combine with, in vLLM's words: "Pipeline parallelism is not
composable with speculative decoding as of `vllm<=0.15.0`", and "Speculative
decoding with a draft models is not supported in `vllm<=0.10.0`". Composing
the two has been requested since 2024 and closed stale (#6911, #10615); PP
also carries a Qwen3.5 bug report closed stale without a fix (#36643).
- Method catalog at v0.25.1: EAGLE, MTP, draft model, PARD, MLP, n-gram, suffix
decoding, hidden-state extraction, a custom proposer class, dynamic
speculative decoding; `dflash` is an accepted `method` value.
## Speculative decoding across the split
llama.cpp chains several implementations behind one comma-separated
`--spec-type` list; the split does not change the chain, only where its passes
run. Mechanisms, chaining laws and drafter KV sizing are
[[DFlash DSpark Speculative Decoding]]'s; only the split interaction is here.
| Method | llama.cpp type | Over ggml-rpc | With vLLM PP | Why |
| --- | --- | --- | --- | --- |
| MTP | `draft-mtp` | works; not documented upstream, but run this way on this fleet's split head with acceptance measured ([[Qwen3.8 PP Deployment]]) | no | Context-backed drafter: it needs a draft context and runs in the head process, so RPC devices are simply more devices under it |
| DFlash | `draft-dflash` | same gate, same status as MTP | no | Same draft-context gate (`ctx_dft`); the drafter's KV lands on whichever device the draft context is built on — the one thing to check on a split |
| ngram-mod | `ngram-mod` | safest of the four: no draft model, no draft context, no remote draft pass | no | Draftless — proposals come from cached token n-grams in the head, so the split adds no device work |
| DSpark | `draft-dspark` | **unproven over RPC** — not a missing feature | no | `draft-dspark` exists upstream; nobody has measured it with the target split across two machines |
**Correction (2026-10-19).** A retired stub in this directory asserted "DSpark
is vLLM-only." False. `draft-dspark` is in upstream llama.cpp — parsed in
`common/arg.cpp`, mapped to `COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK` in
`common/speculative.cpp`, where it carries a Markov head and an anchor-first
block layout and aborts with `DSpark draft has no confidence head: please set
--spec-draft-p-min 0` when the drafter has no confidence head. What is unproven
is running it over RPC, and that is the honest limit of the claim.
The vLLM column is uniform because that statement is blanket: it names pipeline
parallelism, not a method, so n-gram and suffix are excluded as written. vLLM
gives no mechanism with it; "the drafter must sit next to the target's layers,
which a stage split separates" is a reading, not upstream's words.
Upstream likewise documents neither for nor against `--rpc` plus
`--spec-type`. What the implementation does say is structural: every
speculative implementation is built in the head process against the head's
device set, and draftless types need no draft context at all. Routing draft
passes to a remote worker, however, has never been tested.
## Mixed-vendor reality
What works today:
- **One vendor per node.** RPC is backend-agnostic by construction: upstream's
own diagram has a host exposing CUDA0+CUDA1, a host exposing Metal, and a
host exposing CUDA0+CPU, all driven by one main host over TCP. Each node
builds its own backend plus `-DGGML_RPC=ON`; nothing speaks CUDA-to-ROCm.
- **Identical build tag on every node**, worker restarted before head. The
verified version-skew gotcha is [[Qwen3.8 PP Deployment]]'s.
- **Speculation on the head** while the split carries the target (table above).
- **RDMA when both peers have it**, auto-negotiated at the handshake; the size
of the gain is third-party and unverified here.
Unproven or broken:
- **One binary holding both CUDA and ROCm for a same-host layer split**:
reported to compile and then list only CUDA devices (issue #11506,
third-party, unverified). Vulkan is the reported single-binary cross-vendor
path, and single-host placement is [[Llama Cpp Multi-Device Splits]]'s, whose
verdict is that per-tensor placement across a vendor mix never works.
- **vLLM across a CUDA node and a ROCm node**: no known experiment; both ends
run the same backend (recorded 2026-08-14). AMD's own two-Halo clustering
guide is tensor parallelism over RCCL, not a mixed-vendor split.
- **DSpark over RPC**: unproven, see the correction above.
- **The exact NVIDIA-desktop + Strix-Halo-NUC pair has no published
benchmark**, and the third-party mixed-vendor figures it is estimated from
(Metal+CUDA over 10 GbE, two Strix Halos over USB4, a three-card CUDA+ROCm set
over 1 GbE, a Vulkan triple in one box, an AMD four-node Strix cluster) live on
[[Expected Performance]] and [[Ethernet Bandwidth Math]], each named with its
model and hardware. None was re-verified in this pass.
## Security
`ggml-rpc-server` has **no authentication and no encryption**. Upstream's own
warning: the backend is "in a proof-of-concept development stage. As such, the
functionality is fragile and insecure. Never run the RPC server on an open
network or in a sensitive environment!" It binds `127.0.0.1:50052` by default —
safe by accident. Bind a routable address and any host that can reach the port
can allocate memory there and supply the tensors it computes on. Trusted LAN
only, firewalled or tunnelled; never a public address.
## Unverified in this pass
Every item was first recorded 2026-08-14, in the research that preceded this
page. Nothing is silently deleted; none is backed by the sources cited below.
- **RPC protocol version and handshake rule** — recorded as protocol `5.0.0`
with "major must match, server minor must be ≥ client". The README's own
banner prints `Starting RPC server v3.0.0`; the conflict is unresolved.
- **RPC internals** — a hard cap of 16 servers, synchronous graph compute, no
tensor copy between two RPC servers, workers loading tensors on first use, a
hash-keyed cache above roughly 10 MB, asserts on quantized tensors whose rows
are not 512-aligned, and `-rpc` on `llama-bench` (the README names only
`llama-cli` and `llama-server`).
- **vLLM flag surface** — `--pipeline-parallel-size`, `--tensor-parallel-size`,
`--distributed-executor-backend {ray,mp}`, `--nnodes`, `--node-rank`,
`--master-addr`, `--headless`, the `ray start --head` join,
`NCCL_SOCKET_IFNAME`.
- **vLLM mechanics** — one activation handoff per stage boundary per forward
pass, a worked ~512 MB handoff for a hidden-5120 model at batch 8 and 2048
positions, `--max-model-len` unchanged by PP, ~50% idle at pp=2 batch one.
- **vLLM throughput verdicts** for a 27B across two nodes over gigabit-class
links; superseded as an estimate by [[Expected Performance]].
- **Mixed-vendor claims** — a CUDA+HIP build listing only CUDA devices
(#11506), Vulkan as the universal cross-vendor backend, AMD's two-Halo
clustering guide being RCCL tensor parallelism, WiFi costing about ten times
wired, the slowest hop setting the decode pace, RDMA worth a few percent.
- **Draft over RPC** — that the draft model stays on the head, and that routing
draft passes to a remote worker has never been confirmed.
## Citations
- [llama.cpp RPC README](https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md), accessed 2026-10-19 — server usage and device exposure, `-DGGML_RPC=ON` on both sides, the `--rpc host:port` list, proportional weight+KV split and `--tensor-split`, the `-c` local cache, RDMA negotiation and `GGML_RPC_NO_RDMA`, the proof-of-concept and no-authentication warning.
- [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md), accessed 2026-10-19 — split modes (`-sm layer` = pipeline parallelism, KV with the layer owner), `-ts`/`-ngl`/`-ctk`/`-ctv`/`-fit`, the pipeline-versus-tensor summary, the tensor-mode limits.
- [vLLM speculative decoding docs, v0.25.1](https://docs.vllm.ai/en/v0.25.1/features/speculative_decoding/), accessed 2026-10-19 — the pipeline-parallelism incompatibility statement, the draft-model version floor, the method catalog.
- [llama.cpp `common/speculative.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/common/speculative.cpp), accessed 2026-10-19 — the speculative implementation set including `draft-dspark`, its draft-context gate, its confidence-head error.
- vLLM issue #6911, "[Feature]: Combine pipeline parallelism with speculative decoding", opened 2024-07-29, closed not-planned 2025-03-28, accessed 2026-10-19.
- vLLM issue #10615, "[Usage]: Does speculative decoding support pipeline parallelism?", opened 2024-11-25, closed not-planned 2025-04-18, accessed 2026-10-19.
- vLLM issue #36643, "[Bug]: Qwen3.5 does not work with pipeline parallelism", opened 2026-03-10, closed not-planned 2026-09-24, accessed 2026-10-19.
@@ -1,9 +0,0 @@
---
type: reference
title: "Qwen3.6 27B Spec Methods"
description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
tags:
- llm
- pipeline
@@ -1,7 +1,7 @@
---
type: reference
title: Sample Script Set
description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
tags:
- pipeline-parallelism
- sample
@@ -255,7 +255,7 @@ is 30 GiB.
## See Also
- [[llama.cpp RPC]]
- [[Multi-Node Runtimes]]
- [[Expected Performance]]
- [[Ethernet Bandwidth Math]]
- [[Qwen3.6 27B Spec Methods]]
- [[DFlash DSpark Speculative Decoding]]
@@ -28,28 +28,24 @@ graph LR
end
```
- [[llama.cpp RPC]] — the working, production-capable path for two
heterogeneous nodes over ethernet.
- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
multiprocessing and does not compose with speculative decoding yet.
## What the Research Found
- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
(about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
tokens per second on paper; the real limit is latency and RPC overhead.
- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
works today via llama.cpp RPC with separate CUDA and ROCm builds. The
exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
published numbers yet.
- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
## Deployment
- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
`--rpc`, default split is proportional to reported free memory (pin 1,1),
@@ -57,7 +53,7 @@ graph LR
## Sample Scripts
- [[Sample Script Set]] — a two-node llama.cpp RPC split of
- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
the largest context that fits. For educational reading only. Do not run
it in production.
@@ -68,3 +64,9 @@ graph LR
- [[frank-zealot]]
- [[Dankserver System Specs]]
- [[Dankclaw System Specs]]
## Pages
| Page | Owns |
|---|---|
| [[Multi-Node Runtimes]] | How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts. |
@@ -1,9 +0,0 @@
---
type: reference
title: "llama.cpp RPC"
description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,9 +0,0 @@
---
type: reference
title: "vLLM Pipeline Parallelism"
description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,3 +1,13 @@
---
type: "howto"
title: "Quantization Production"
description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
tags:
- llm
- quantization
timestamp: 2026-10-19
---
# Quantization Production — making our own quants, per card and runtime
**Research page (2026-10-08).** Question: how do WE produce quantizations
@@ -1,7 +1,7 @@
---
type: reference
title: RDNA4 Container Serving
description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: RDNA4 MoE Expert Offload VRAM Budget
description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
tags:
- llm
timestamp: 2026-09-03
@@ -1,6 +1,6 @@
---
type: reference
description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
tags:
- llm
timestamp: 2026-09-03
@@ -1,7 +1,7 @@
---
type: reference
title: Tcclaviger Fork Ops
description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
timestamp: 2026-10-05T00:00:00-07:00
---
+1 -1
View File
@@ -1,6 +1,6 @@
---
type: reference
description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
description: "Locator for every vLLM page in this vault."
tags:
- llm
- vllm
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
tags:
- dankclaw
- iommu
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Frontier MoE on Dankserver — what fits, on which runtime
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
tags:
- dankserver
- r9700
@@ -1,7 +1,7 @@
---
type: reference
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot.
description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
tags:
- llamacpp
- cuda
@@ -9,6 +9,7 @@ tags:
- moe
- ornith
- mtp
- retired-endpoint
timestamp: 2026-08-24
---
@@ -123,4 +124,4 @@ llama-server \
- [[deployment]] — deploy and update process
- [[Dankserver System Specs]] — the box
- [[frank-redo-2026-08-17]] — fleet topology history
- [[frank-redo-2026-08-17]] — fleet topology histor
+1 -1
View File
@@ -1,5 +1,5 @@
---
type: guide
type: howto
title: LLM Deployment
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
tags:
+7 -3
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
tags:
- llm
- frank-archon
@@ -11,6 +11,7 @@ tags:
- r9700
- tp2
- expert-offload
- endpoint
timestamp: 2026-10-18
---
@@ -139,8 +140,10 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
pages are **not** superseded here: [[frank-medium]] is the locator for that
endpoint's whole shape history, including the unresolved end date of the
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
## Laws learned here
@@ -438,3 +441,4 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
[[frank-medium-radiance-mxfp4]], [[LLM/index]].
---
+3 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
tags:
- llm
- frank-dragoon
@@ -9,6 +9,7 @@ tags:
- strix-halo
- gfx1151
- halogen
- endpoint
timestamp: 2026-10-18
---
@@ -234,3 +235,4 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
---
@@ -2,7 +2,7 @@
timestamp: 2026-09-17
type: reference
title: frank-medium — Batch and Ubatch Sweep
description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides.
description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
tags:
- benchmark
- llamacpp
@@ -11,6 +11,7 @@ tags:
- dankserver
- layer-split
---
- retired-endpoint
# frank-medium — Batch and Ubatch Sweep
@@ -222,4 +223,5 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
difference: with no KV to attend over, per-pass overhead dominates and a
bigger micro-batch wins; at 64k of resident KV it does not.
- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
- [[Dankserver System Specs]] — hardware inventory
- [[Dankserver System Specs]] — hardware inventory---
timestamp: 2026-09-17
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
tags:
- frank-medium
- dankserver
@@ -12,6 +12,7 @@ tags:
- gfx1201
- r9700
- layer-split
- retired-endpoint
timestamp: 2026-09-20
---
@@ -730,3 +731,4 @@ fleet.
- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
why the logical batch is free, and the harness traps
---
@@ -1,7 +1,7 @@
---
type: runbook
type: howto
title: frank-medium Model Swap
description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path."
tags:
- frank-medium
- dankserver
@@ -9,6 +9,7 @@ tags:
- vllm-radiance
- huggingface
- runbook
- retired-endpoint
timestamp: 2026-09-20
---
@@ -155,9 +156,10 @@ back:
2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
3. Re-run the capability battery.
The 2026-09-20 rollback target is the MXFP4 checkpoint at
`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single
safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
The 2026-09-20 rollback target is the MXFP4 checkpoint with
`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that
payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a
local file.
## 2026-09-20 swap record (MXFP4 → FP8)
@@ -184,3 +186,4 @@ safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
vs 19.4 GiB).
---
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium vllm-radiance MXFP6 Runbook
description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
tags:
- frank-medium
- dankserver
@@ -14,6 +14,7 @@ tags:
- rccl
- pcie-atomics
- quadlet
- retired-endpoint
timestamp: 2026-09-21
---
@@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
image back to `localhost/radiance-vllm:0.9.3` (re-pull
`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
486,464 tokens with bf16 KV and held the full 262,144 window).
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
@@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
with growing history) shows what happens at that limit: turns alternate between
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
and must be prefilled again, because two families per card plus their cached
prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
both replicas steady — and per-family pinning kept the fast turns on the replica
that holds the prefix.
prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
turns on the replica that holds the prefix.
Read that as a capacity budget, not a defect: two concurrent large-context
sessions per card are comfortable, three or more of that size trade turns
@@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here.
the same image and kernel 7.0.0-30 failing identically.
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
this page compares against.
---
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium ROCmFPX Runbook
description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201).
description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required."
tags:
- frank-medium
- dankserver
@@ -10,6 +10,7 @@ tags:
- mtp
- gfx1201
- r9700
- retired-endpoint
timestamp: 2026-09-15
---
+65
View File
@@ -0,0 +1,65 @@
---
type: reference
title: frank-medium (retired) — the Dankserver endpoint before frank-archon
description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree."
timestamp: 2026-10-19
tags:
- llm
- dankserver
- frank-medium
- retired
- retired-endpoint
---
# frank-medium (retired) — the Dankserver endpoint before frank-archon
`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live
endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every
shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon`
name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** —
which era page holds which payload's evidence — and carries no measurements: every
number stays in the era page that measured it.
## Eras
Windows are quoted from each era page's own banner; no date here is derived.
| Window | Runtime | Payload | Shape | Page | Rollback |
|---|---|---|---|---|---|
| 2026-09-15 → 2026-09-16 | llama.cpp, ROCmFPX fork pinned at commit `36e9acd` | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) | single card; the fork is the only loader | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork |
| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags |
| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape |
| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4, then MXFP6 (`paroquant_mxfp6`) with bf16 KV | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` |
`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run
on the layer-split pair at its deployed shape, and its numbers belong to that
campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of
the 2026-09-20 swap procedure — while the procedure itself is owned by
[[Model Swap and Rollback]].
## Laws
None restated. Each era page carries its own laws and links them to the technique
page that owns the mechanism ([[PCIe Link and Atomics]],
[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]],
[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]).
## Open questions
Two date claims in the era pages do not reconcile, and both are left standing rather
than silently corrected:
- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the
next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The
MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which
one directly followed the ROCmFPX fork cannot be settled from the repo.
- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while
[[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded
fortnight on that endpoint or a stale banner; nothing else in the repo covers
2026-09-26 → 10-10 for port 7770.
## Files
No repo files own this endpoint: its units, env and serve scripts were renamed to
`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds
retired serve scripts from these eras as reference only.
+2 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went."
tags:
- flm
- fastflowlm
@@ -10,6 +10,7 @@ tags:
- dankclaw
- frank-npu
- retired
- retired-endpoint
timestamp: 2026-09-11
---
@@ -1,9 +1,10 @@
---
type: reference
description: "This page records the full redo of the inference fleet on 2026-08-17."
description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way."
tags:
- llm
- frank
- retired-endpoint
timestamp: 2026-09-03
---
@@ -223,3 +224,4 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17
evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
the main TTFT lever for pi, which re-sends the same system prompt + tool
schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
---
+2 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV"
description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
tags:
- llm
- frank-zealot
@@ -9,6 +9,7 @@ tags:
- llama-cpp
- gguf
- rtx3090
- endpoint
timestamp: 2026-10-18
---
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: FreeToken Benchmark Sweep — Dankserver RTX 3090
description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2.
description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime."
tags:
- freetoken
- benchmark
@@ -1,7 +1,7 @@
---
type: reference
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required."
tags:
- ikllama
- llamacpp
@@ -11,7 +11,8 @@ tags:
- ornith
- kv-cache
- hadamard
timestamp: 2026-08-26
- retired-endpoint
timestamp: 2026-10-19
---
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
@@ -259,4 +260,4 @@ spec / `--spec-type` flags. The model file is not changed either way.
build flags incl. `GGML_CUDA_FA_ALL_QUANTS` and `GGML_IQK_FA_ALL_QUANTS`
- [ik_llama.cpp serverAppConfigs-docs/parameters.md](https://github.com/ikawrakow/ik_llama.cpp/blob/main/docs/parameters.md) —
q8_KV (PR 208), K/V Hadamard options, `-cuda graphs=0` workaround, and
`--split-mode graph` MoE note (PR 1080)
`--split-mode graph` MoE note (PR 1080)
+54 -53
View File
@@ -9,37 +9,33 @@ each of those owns exactly that one thing. The incidents behind the laws are in
| Page | Owns |
|---|---|
| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |
## Retired endpoints
| Page | Owns |
|---|---|
| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |
## Pipeline Parallelism (series)
| Page | Owns |
|---|---|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |
## Techniques, hardware, trials
@@ -47,43 +43,48 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|---|---|
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
| [[Quantization Production]] | — |
| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
| [[swift15-campaign-worklog]] | — |
| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
## Pages
| Page | Owns |
|---|---|
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |
@@ -1,10 +1,11 @@
---
type: reference
description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw."
description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it."
tags:
- llm
- rocm
- frank
- retired-endpoint
timestamp: 2026-09-15
---
@@ -239,3 +240,4 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
`gl=45`. See the `set_rows` build bug section above.
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
retired the same day for the AGENT model). See [[frank-dragoon]].
---
@@ -1,9 +1,9 @@
---
type: reference
description: "`bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`."
description: "Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict."
tags:
- llm
timestamp: 2026-09-03
timestamp: 2026-10-19
---
# Ling-3.0-flash Q6_K Trial on Dankclaw
@@ -109,6 +109,13 @@ re-enable `frank-dragoon-ling` from the repo unit
- The heap is effectively full (98.7 of 111.2 GiB). Do not raise the
context much without checking `--list-devices` free MiB.
## The other path: vLLM-ROCm (never shipped)
A deployment plan existed for `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native`
(INT4 W4A16, ~62 GB) served by vLLM-ROCm on Dankclaw. It never shipped; the
llama.cpp Q6_K fork trial above is what actually ran. The decision table the
plan carried is in git history.
## See Also
- [[frank-dragoon]]
@@ -1,22 +0,0 @@
---
type: guide
description: "This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on"
tags:
- llm
- vllm
- rocm
timestamp: 2026-09-03
---
# Ling-3.0-flash on Dankclaw — vLLM-ROCm Deployment Plan (never shipped)
This page was the plan for a vLLM-ROCm deployment of
`jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on
Dankclaw. The deployment never shipped. The actual Ling work used the
llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]].
The trial service (`frank-dragoon-ling`) was retired and disabled under the boot
policy on 2026-08-17, in the same redo that renamed `frank-dragoon` to the
Qwen3.8-27B PP head ([[frank-redo-2026-08-17]]). Two fleet generations later the
name belongs to the halogen flash-server endpoint ([[frank-dragoon]]). See git
history for the full decision table.
@@ -1,6 +1,6 @@
---
type: reference
description: "This page records the speculative-decoding benchmark history and the"
description: "The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out."
tags:
- llm
- speculative
@@ -1,6 +1,6 @@
---
type: reference
description: "This page answers one question: **how do we make Qwen3.8-27B serve faster on"
description: "How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought."
tags:
- llm
timestamp: 2026-09-03
@@ -1,3 +1,13 @@
---
type: "reference"
title: "Swift1.5 Campaign Worklog"
description: "Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm."
tags:
- llm
- campaign
timestamp: 2026-10-19
---
# Swift1.5 w4dn8 Campaign — Work Log
Campaign branch `swift15-radiance-w4dn8`. Goal: convert
@@ -1,7 +1,7 @@
---
type: benchmark plan
type: reference
title: frank-dragoon ubatch sweep
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
description: "The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness."
tags:
- llama.cpp
- rocm
@@ -1,7 +1,7 @@
---
type: reference
title: vLLM Boot Forensics and Config Interrogation
description: Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone), classifying the failure, and interrogating a live endpoint's true config and client contracts.
description: "Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config."
tags: [vllm, forensics, journalctl, podman, config, contracts]
timestamp: 2026-10-05T00:00:00-07:00
---

Some files were not shown because too many files have changed in this diff Show More