docs(vault): one fact, one owner — repair the doubled pages, single-source the facts
Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Agent Docs and Edits
|
||||
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
|
||||
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
|
||||
tags: [agents, docs, markdown, hashline, edit-recovery]
|
||||
timestamp: 2026-10-18T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Command Center Dotfiles
|
||||
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
|
||||
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
|
||||
tags: [dotfiles, omp, deploy, rsync, blankpad]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: OMP Harness
|
||||
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
|
||||
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
|
||||
tags: [omp, harness, compaction, models-yml, context, samplers]
|
||||
timestamp: 2026-10-18T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,23 +1,12 @@
|
||||
---
|
||||
type: index
|
||||
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
|
||||
tags: [agent, index]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
# Agent
|
||||
|
||||
Pages about the agent's own tooling and context — how omp is configured, how
|
||||
docs get edited safely, and how the agent's home is deployed.
|
||||
|
||||
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
|
||||
`compat.extraBody`, preamble and skill-library audits
|
||||
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
|
||||
hashline edit recovery
|
||||
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
|
||||
transport, override packages
|
||||
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
|
||||
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
|
||||
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
|
||||
|
||||
## Related
|
||||
|
||||
- [[Testing Policy]] — the seams these pages' checks ride
|
||||
- [LLM](../LLM/index.md) — the fleet these agents operate
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Container Network Namespaces
|
||||
description: Which local services are addressable from which Podman network namespace on
|
||||
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
|
||||
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
|
||||
tags:
|
||||
- networking
|
||||
- podman
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Deploy State Triage
|
||||
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
|
||||
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
|
||||
tags: [deployment, deploy-state, systemd, podman, triage]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
type: guide
|
||||
type: howto
|
||||
title: Deployment Flow
|
||||
description: Service-scoped Git deployment for the two homelab hosts.
|
||||
tags:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
|
||||
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
|
||||
tags:
|
||||
- architecture
|
||||
- e2e
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Quadlet Lifecycle
|
||||
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
|
||||
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
|
||||
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Stale Signal Audit
|
||||
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
|
||||
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
|
||||
tags: [testing, observability, audit, signals, gates]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: howto
|
||||
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
|
||||
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
|
||||
tags:
|
||||
- dankclaw
|
||||
- comfyui
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
|
||||
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
|
||||
tags:
|
||||
- dankclaw
|
||||
- tts
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
|
||||
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
|
||||
tags:
|
||||
- endpoint
|
||||
- dankclaw
|
||||
- tts
|
||||
- voice
|
||||
- quadlet
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Frank TTS
|
||||
@@ -202,3 +204,4 @@ harness bundle and measured the day of the flip:
|
||||
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
||||
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
||||
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
|
||||
---
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Gitea
|
||||
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
|
||||
tags:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Hermes-WebUI (Hermex server)
|
||||
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
|
||||
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
|
||||
tags:
|
||||
- hermes
|
||||
- hermes-webui
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Marreta on Dankclaw
|
||||
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
|
||||
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
|
||||
tags:
|
||||
- dankclaw
|
||||
- marreta
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
|
||||
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
|
||||
tags:
|
||||
- dankclaw
|
||||
- open-notebook
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
|
||||
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
|
||||
tags:
|
||||
- dankclaw
|
||||
- seerr
|
||||
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
|
||||
authentik's own user UI instead of Seerr. Both backfilled live: the three
|
||||
standard flows copied from a known-good provider, plus an application
|
||||
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
|
||||
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
|
||||
provisioning rules this incident established.
|
||||
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
|
||||
two provisioning rules this incident established.
|
||||
|
||||
## *arr wiring
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Dankclaw Swap Configuration
|
||||
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
|
||||
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
|
||||
tags:
|
||||
- dankclaw
|
||||
- host
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
|
||||
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
|
||||
tags:
|
||||
- dankclaw
|
||||
- frank-tts
|
||||
|
||||
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
|
||||
|
||||
## Security Advisories
|
||||
|
||||
| Version | CVE | Issue |
|
||||
|---------|-----|-------|
|
||||
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
|
||||
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
|
||||
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
|
||||
| Version | Advisory | CVE | Issue |
|
||||
|---------|----------|-----|-------|
|
||||
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
|
||||
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
|
||||
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
|
||||
|
||||
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
|
||||
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
|
||||
- [[Dankserver Stack]]
|
||||
- [[Caddy]] — TLS termination and authentication in front
|
||||
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
|
||||
|
||||
## Citations
|
||||
|
||||
- GitHub Security Advisories, `9001/copyparty` —
|
||||
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
|
||||
each row's patched version read from `first_patched_version` in the
|
||||
`https://api.github.com/advisories/<GHSA-id>` response.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Dankserver (192.168.0.235) runs the media and library stack."
|
||||
description: "Service inventory and routing overview for the media and library host."
|
||||
tags:
|
||||
- dankserver
|
||||
timestamp: 2026-09-03
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Jellyfin
|
||||
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
|
||||
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
|
||||
tags: [jellyfin, dankserver, media-server]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
|
||||
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
|
||||
tags:
|
||||
- dankserver
|
||||
- romm
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: MariaDB (RomM Backend)
|
||||
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
|
||||
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
|
||||
tags: [mariadb, romm, dankserver]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Redis (RomM Cache / Queue)
|
||||
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
|
||||
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
|
||||
tags: [redis, romm, dankserver]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Noctalia and Niri
|
||||
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
|
||||
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
|
||||
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "*arr Stack"
|
||||
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
|
||||
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
|
||||
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
|
||||
timestamp: 2026-09-04T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
|
||||
|
||||
## *arr Stack
|
||||
|
||||
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
|
||||
- [[sonarr]] — TV show management
|
||||
- [[radarr]] — movie management
|
||||
- [[lidarr]] — music management
|
||||
- [[readarr]] — ebook/audiobook management
|
||||
- [[prowlarr]] — indexer management
|
||||
- [[qbittorrent]] — torrent download client
|
||||
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
|
||||
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
|
||||
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
|
||||
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Prowlarr
|
||||
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
|
||||
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
|
||||
tags: [prowlarr, arr-stack, frankbook-pro]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: qBittorrent
|
||||
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
|
||||
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
|
||||
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
---
|
||||
type: reference
|
||||
title: Readarr
|
||||
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
|
||||
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
|
||||
tags: [readarr, arr-stack, frankbook-pro]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
# Readarr
|
||||
|
||||
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
|
||||
> metadata became unusable and the community migration to Open Library stalled.
|
||||
> There is no upstream development. Alternatives: the Bookshelf community fork,
|
||||
> or the Calibre content server. This instance still runs on a pinned image
|
||||
> tag; see below.
|
||||
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
|
||||
> Readarr: its metadata became unusable and the community migration to Open
|
||||
> Library stalled. There is no upstream development — the GitHub repository is
|
||||
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
|
||||
> community fork, or the Calibre content server. This instance still runs on a
|
||||
> pinned image tag; see below.
|
||||
|
||||
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
|
||||
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
|
||||
@@ -69,9 +70,9 @@ Restart=always
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
|
||||
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
|
||||
`:latest` does not exist for this image on LSIO.
|
||||
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
|
||||
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
|
||||
because `:latest` does not exist for this image on LSIO.
|
||||
- Calibre integration is folder-based, not API-based. Readarr drops files into
|
||||
the books root which [[Calibre-Web]] serves.
|
||||
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
|
||||
@@ -81,3 +82,10 @@ Restart=always
|
||||
|
||||
- [[ARR-Stack]] — full *arr stack documentation
|
||||
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
|
||||
|
||||
## Citations
|
||||
|
||||
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
|
||||
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
|
||||
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
|
||||
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
|
||||
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
|
||||
tags:
|
||||
- infrastructure
|
||||
timestamp: 2026-09-17
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Authentik Outpost Reference
|
||||
|
||||
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
|
||||
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
|
||||
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
|
||||
self-serve user-settings flow, and the live database shape. What the service is,
|
||||
how it is deployed, wired and operated is [[Authentik]]; the three core
|
||||
containers are [[Authentik Processes]].
|
||||
|
||||
## Live proxy-provider inventory
|
||||
|
||||
Live provider inventory (read from the `authentik-postgres` container with
|
||||
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
||||
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
||||
members in `authentik_outposts_outpost_providers`), all with
|
||||
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
||||
UUIDs** (`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent`,
|
||||
`default-provider-invalidation-flow`), and each backed by an Application
|
||||
(the `slug` column below):
|
||||
|
||||
| Provider pk | name | external_host | app slug |
|
||||
|---|---|---|---|
|
||||
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
||||
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
||||
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
||||
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
||||
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
||||
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
||||
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
||||
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
||||
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
||||
|
||||
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
||||
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
|
||||
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
||||
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
||||
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
||||
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
||||
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
||||
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
||||
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
||||
below rejects), and pk 14 `Open WebUI`
|
||||
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
||||
`external_host` (the runbook-recipe style) and bare origins both intercept
|
||||
correctly — the outpost matches on hostname.
|
||||
|
||||
## Provisioning rules for proxy providers
|
||||
|
||||
Two provisioning rules, each verified live by an outage:
|
||||
|
||||
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
||||
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
||||
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
||||
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
||||
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
||||
forever, and **everything behind the outpost dies at the next outpost
|
||||
reload** — including hosts whose providers were fine (verified 2026-09-02
|
||||
when three providers provisioned without any flows took down every guarded
|
||||
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
||||
`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent` (the implicit-consent
|
||||
authorization flow, `default-invalidation-flow`) and the retry loop
|
||||
recovers on its next 3-second cycle. The silent killer: while the outpost
|
||||
process keeps its last good in-memory config, everything works — the
|
||||
outage only surfaces on restart/reload.
|
||||
2. **Every proxy provider needs an attached Application.** The outpost
|
||||
resolves the post-login redirect through the provider's application; a
|
||||
provider with no app completes the login (authentik session established)
|
||||
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
||||
the app. No error is logged anywhere — the SSO "works" and the app never
|
||||
appears. Create the Application with `provider_id` set (the UI does this
|
||||
automatically; raw API/ORM drives must do it explicitly).
|
||||
|
||||
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
||||
|
||||
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
||||
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
||||
implementation diverges from upstream in ways that bit us; all verified live:
|
||||
|
||||
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
||||
create the single-table-inheritance `Application` row; without it the flow
|
||||
executor can never complete an authorize request. Create both rows.
|
||||
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
||||
contains `"authorization_code"` (enum value with underscore), the token
|
||||
endpoint rejects every exchange with `invalid_request`.
|
||||
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
||||
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
|
||||
POST-body client secrets are not accepted — authenticate the confidential
|
||||
client with `Authorization: Basic base64(client_id:client_secret)`.
|
||||
- **Provider resolution is by `client_id` query param** on the global
|
||||
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
||||
this build. Token/userinfo live at `/application/o/token/` and
|
||||
`/application/o/userinfo/?api=1`.
|
||||
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
||||
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
||||
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
||||
becomes the *expression source* instead of the context; every call raises a
|
||||
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
||||
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
||||
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
|
||||
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
||||
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
||||
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
||||
build until the evaluator is patched or upgraded. Clients consuming this
|
||||
provider must key identity off `sub`.
|
||||
- Scope mappings for this provider exist — `grafana-email-scope` and
|
||||
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
||||
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
||||
the evaluator bug is fixed; no config change is needed then.
|
||||
|
||||
## LDAP provider objects (provisioned via `ak shell` ORM)
|
||||
|
||||
The LDAP outpost's own deployment — unit, image digest, published ports, token
|
||||
env var — is in [[Authentik]]. The objects it serves:
|
||||
|
||||
Read from the live database (read-only `SELECT`s against the
|
||||
`authentik-postgres` container, 2026-10-18):
|
||||
|
||||
| Object | Value |
|
||||
|---|---|
|
||||
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
||||
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
||||
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
||||
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
|
||||
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
||||
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
||||
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
|
||||
|
||||
Hard-won rules (the first is now contested by the live database):
|
||||
|
||||
- **Token intent: contested, do not trust either value blindly.** An earlier
|
||||
revision asserted that a bind token with `intent=app_password` is rejected by
|
||||
the flow's password stage with `Invalid password` and only
|
||||
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
|
||||
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
|
||||
wrong or the current Jellyfin bind path is not that token. Nothing observable
|
||||
settles it today: the LDAP outpost's retained log holds only its 5-minute
|
||||
`initialized direct binder` / `Update providers` config cycles and no bind
|
||||
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
||||
requires the bind secret)*. Re-verify with a real bind before changing the
|
||||
token's intent either way.
|
||||
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
||||
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
||||
matches zero entries and the plugin logs `Found no users matching X`.
|
||||
The permission is object-level on the provider; grant it via an RBAC role
|
||||
and retest.
|
||||
- `ak shell` import path for the provider model in 2026.8:
|
||||
`from authentik.providers.ldap.models import LDAPProvider`.
|
||||
*(unverified 2026-10-18: importing it outside a configured Django shell fails
|
||||
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
||||
auth service for this check)*
|
||||
|
||||
## Self-serve user-settings flow (password change)
|
||||
|
||||
Users change their own account details through the `default-user-settings-flow`
|
||||
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
||||
target of the homepage footer's "account" link. It is a `stage_configuration`
|
||||
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
||||
user.
|
||||
|
||||
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
||||
insertion order):
|
||||
|
||||
| order | stage | type | what it does |
|
||||
|---|---|---|---|
|
||||
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
||||
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
||||
|
||||
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
||||
writes whatever the preceding `PromptStage` collected: for the key `password`
|
||||
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
||||
is a `password` field on the `default-user-settings` prompt
|
||||
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
||||
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
||||
`validate_password` runs `authenticate()` against the backends to check the
|
||||
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
||||
2026-09-25 and removed the same day for exactly this reason.)
|
||||
|
||||
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
||||
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
||||
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
|
||||
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
||||
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
||||
|
||||
**A blank password must not set an empty password.** `set_password("")` would
|
||||
make the account log in with no password, so a validation policy
|
||||
(`default-user-settings-no-empty-password`, bound to the prompt via
|
||||
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
||||
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
||||
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
||||
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
||||
invalid).
|
||||
|
||||
## Live database shape
|
||||
|
||||
For live DB truth prefer the postgres container, and it works without any
|
||||
credential — the local socket is trusted, so a read-only audit is just
|
||||
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
||||
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
||||
before guessing: 2026.x renamed both tables and columns, and the models are
|
||||
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
||||
three `*_flow_id` columns; the proxy table is
|
||||
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
||||
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
||||
table in 2026.8), the LDAP table is
|
||||
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
|
||||
`authentik_core_application` attaches to a provider through `provider_id`
|
||||
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
||||
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
||||
is the same story from the ORM side: `type` lives in the subclass tables.
|
||||
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
||||
failure modes
|
||||
- [[Authentik Processes]] — the postgres, server and worker containers
|
||||
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
|
||||
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
|
||||
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
- postgres
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Authentik Processes
|
||||
|
||||
The three core containers of the [[Authentik]] stack on Dankclaw. The two
|
||||
outpost units are not covered here — they live on [[Authentik]] and
|
||||
[[Authentik Outpost Reference]]. Server and worker run the same image; the
|
||||
`command` selects the role (`server` / `worker`). Data persists under
|
||||
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
|
||||
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
|
||||
standardization; the retired stubs this page replaces recorded the pre-move
|
||||
paths `/home/dank/authentik/data` (server and worker) and
|
||||
`/home/dank/authentik/postgres` (database).
|
||||
|
||||
## authentik-postgres
|
||||
|
||||
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
|
||||
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
|
||||
PostgreSQL in 2025.10 (Redis removed).
|
||||
|
||||
- **Image:** `docker.io/library/postgres:16-alpine`
|
||||
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
|
||||
volumes)
|
||||
- **Health:** `pg_isready -d authentik -U authentik`
|
||||
- **Retired wording:** the stub called this a "Compose service … in
|
||||
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
|
||||
- The side-by-side with `gitea-postgres` (network, published ports, secret
|
||||
source, boot persistence) is owned by [[Postgres]].
|
||||
|
||||
## authentik-server
|
||||
|
||||
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
||||
It also serves the login page at `https://auth.franksdank.cloud`.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
|
||||
(`authentik-server.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
|
||||
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
|
||||
live here
|
||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
||||
worker)
|
||||
- **Health:** recorded by the retired stub as
|
||||
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
|
||||
owns the live probe.
|
||||
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
|
||||
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
|
||||
integration.
|
||||
|
||||
## authentik-worker
|
||||
|
||||
The Authentik worker — background tasks: event processing, provider
|
||||
synchronization, scheduled flows, and certificate renewal.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
|
||||
(`authentik-worker.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
|
||||
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
|
||||
the data dir)
|
||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
||||
server)
|
||||
- **No port** — the worker has no HTTP endpoint.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
||||
failure modes
|
||||
- [[Authentik Outpost Reference]] — provider, flow and token configuration
|
||||
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
|
||||
@@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
|
||||
they handle their own session. Grafana is the exception that is both: it is a
|
||||
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
|
||||
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
||||
`dankclaw/Caddyfile:374-377`; live provider table below).
|
||||
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
|
||||
|
||||
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
|
||||
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
|
||||
@@ -35,6 +35,13 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
||||
pre-quadlet era, retired.
|
||||
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
|
||||
|
||||
**Split off from this page:** [[Authentik Processes]] owns the per-container
|
||||
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
|
||||
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
|
||||
the live proxy-provider inventory, the proxy-provider provisioning rules, the
|
||||
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
|
||||
design, and the live database shape.
|
||||
|
||||
## Why Authentik
|
||||
|
||||
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
|
||||
@@ -100,23 +107,10 @@ contract:
|
||||
The chain terminates there — nothing dials back into Authentik from
|
||||
`:8017`.
|
||||
|
||||
## The three compose services
|
||||
## The stack's processes
|
||||
|
||||
| Service | Role | Image |
|
||||
|---|---|---|
|
||||
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
|
||||
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||
|
||||
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
|
||||
moved to PostgreSQL in 2025.10 (Redis removed).
|
||||
- **Server** and **worker** run the same image; the `command` selects the role
|
||||
(`server` / `worker`).
|
||||
- The worker runs as `0:0` (rootless: container root maps to the host user that
|
||||
owns the data dir).
|
||||
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
|
||||
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
|
||||
in the 2026-09-01 quadlet standardization).
|
||||
Per-container facts — unit path, image and role, port, run-as, data directory,
|
||||
health probe — are owned by [[Authentik Processes]].
|
||||
|
||||
## Caddy integration
|
||||
|
||||
@@ -142,104 +136,9 @@ routing table, and the Authentik proxy-provider list must be the same set of
|
||||
hosts. A guarded host with no provider 404s at login time; a provider with no
|
||||
DNS record breaks off-LAN clients.
|
||||
|
||||
Live provider inventory (read from the `authentik-postgres` container with
|
||||
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
||||
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
||||
members in `authentik_outposts_outpost_providers`), all with
|
||||
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
||||
UUIDs** (`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent`,
|
||||
`default-provider-invalidation-flow`), and each backed by an Application
|
||||
(the `slug` column below):
|
||||
|
||||
| Provider pk | name | external_host | app slug |
|
||||
|---|---|---|---|
|
||||
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
||||
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
||||
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
||||
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
||||
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
||||
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
||||
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
||||
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
||||
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
||||
|
||||
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
||||
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
|
||||
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
||||
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
||||
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
||||
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
||||
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
||||
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
||||
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
||||
below rejects), and pk 14 `Open WebUI`
|
||||
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
||||
`external_host` (the runbook-recipe style) and bare origins both intercept
|
||||
correctly — the outpost matches on hostname.
|
||||
|
||||
Two provisioning rules, each verified live by an outage:
|
||||
|
||||
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
||||
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
||||
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
||||
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
||||
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
||||
forever, and **everything behind the outpost dies at the next outpost
|
||||
reload** — including hosts whose providers were fine (verified 2026-09-02
|
||||
when three providers provisioned without any flows took down every guarded
|
||||
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
||||
`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent` (the implicit-consent
|
||||
authorization flow, `default-invalidation-flow`) and the retry loop
|
||||
recovers on its next 3-second cycle. The silent killer: while the outpost
|
||||
process keeps its last good in-memory config, everything works — the
|
||||
outage only surfaces on restart/reload.
|
||||
2. **Every proxy provider needs an attached Application.** The outpost
|
||||
resolves the post-login redirect through the provider's application; a
|
||||
provider with no app completes the login (authentik session established)
|
||||
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
||||
the app. No error is logged anywhere — the SSO "works" and the app never
|
||||
appears. Create the Application with `provider_id` set (the UI does this
|
||||
automatically; raw API/ORM drives must do it explicitly).
|
||||
|
||||
|
||||
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
||||
|
||||
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
||||
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
||||
implementation diverges from upstream in ways that bit us; all verified live:
|
||||
|
||||
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
||||
create the single-table-inheritance `Application` row; without it the flow
|
||||
executor can never complete an authorize request. Create both rows.
|
||||
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
||||
contains `"authorization_code"` (enum value with underscore), the token
|
||||
endpoint rejects every exchange with `invalid_request`.
|
||||
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
||||
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
|
||||
POST-body client secrets are not accepted — authenticate the confidential
|
||||
client with `Authorization: Basic base64(client_id:client_secret)`.
|
||||
- **Provider resolution is by `client_id` query param** on the global
|
||||
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
||||
this build. Token/userinfo live at `/application/o/token/` and
|
||||
`/application/o/userinfo/?api=1`.
|
||||
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
||||
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
||||
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
||||
becomes the *expression source* instead of the context; every call raises a
|
||||
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
||||
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
||||
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
|
||||
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
||||
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
||||
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
||||
build until the evaluator is patched or upgraded. Clients consuming this
|
||||
provider must key identity off `sub`.
|
||||
- Scope mappings for this provider exist — `grafana-email-scope` and
|
||||
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
||||
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
||||
the evaluator bug is fixed; no config change is needed then.
|
||||
The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
|
||||
the two provisioning rules the outpost enforces, and this fork's OAuth2
|
||||
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
|
||||
|
||||
## LDAP outpost (Jellyfin auth, live 2026-09-01)
|
||||
|
||||
@@ -267,44 +166,9 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
|
||||
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
|
||||
from the host `dankclaw/.env` (git-ignored).
|
||||
|
||||
### Authentik-side objects (provisioned via `ak shell` ORM)
|
||||
|
||||
Read from the live database (read-only `SELECT`s against the
|
||||
`authentik-postgres` container, 2026-10-18):
|
||||
|
||||
| Object | Value |
|
||||
|---|---|
|
||||
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
||||
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
||||
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
||||
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
|
||||
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
||||
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
||||
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
|
||||
|
||||
Hard-won rules (the first is now contested by the live database):
|
||||
|
||||
- **Token intent: contested, do not trust either value blindly.** This page
|
||||
long asserted that a bind token with `intent=app_password` is rejected by the
|
||||
flow's password stage with `Invalid password` and only `intent=verification`
|
||||
passes. The live token for `svc-jellyfin-ldap` is `app_password`
|
||||
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
|
||||
current Jellyfin bind path is not that token. Nothing observable settles it
|
||||
today: the LDAP outpost's retained log holds only its 5-minute
|
||||
`initialized direct binder` / `Update providers` config cycles and no bind
|
||||
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
||||
requires the bind secret)*. Re-verify with a real bind before changing the
|
||||
token's intent either way.
|
||||
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
||||
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
||||
matches zero entries and the plugin logs `Found no users matching X`.
|
||||
The permission is object-level on the provider; grant it via an RBAC role
|
||||
and retest.
|
||||
- `ak shell` import path for the provider model in 2026.8:
|
||||
`from authentik.providers.ldap.models import LDAPProvider`.
|
||||
*(unverified 2026-10-18: importing it outside a configured Django shell fails
|
||||
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
||||
auth service for this check)*
|
||||
The objects behind it — LDAP provider, bind DN, service account and bind token,
|
||||
RBAC grant, groups, application slug — and the hard-won bind rules are in
|
||||
[[Authentik Outpost Reference]].
|
||||
|
||||
## Configuration (secrets)
|
||||
|
||||
@@ -340,7 +204,8 @@ every outpost session (see the ORM playbook).
|
||||
Applications** → add an application + a **Proxy** provider whose
|
||||
`external_host` is exactly the browser-facing origin
|
||||
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
|
||||
flows set (see the provisioning pitfall above). Attach the provider to the
|
||||
flows set (see the provisioning rules in
|
||||
[[Authentik Outpost Reference]]). Attach the provider to the
|
||||
embedded outpost.
|
||||
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
|
||||
the app with the correct redirect URI, and paste the `client_id` /
|
||||
@@ -373,47 +238,6 @@ every outpost session (see the ORM playbook).
|
||||
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
|
||||
Authentik migrates the schema on startup.
|
||||
|
||||
## Self-serve user-settings flow (password change)
|
||||
|
||||
Users change their own account details through the `default-user-settings-flow`
|
||||
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
||||
target of the homepage footer's "account" link. It is a `stage_configuration`
|
||||
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
||||
user.
|
||||
|
||||
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
||||
insertion order):
|
||||
|
||||
| order | stage | type | what it does |
|
||||
|---|---|---|---|
|
||||
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
||||
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
||||
|
||||
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
||||
writes whatever the preceding `PromptStage` collected: for the key `password`
|
||||
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
||||
is a `password` field on the `default-user-settings` prompt
|
||||
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
||||
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
||||
`validate_password` runs `authenticate()` against the backends to check the
|
||||
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
||||
2026-09-25 and removed the same day for exactly this reason.)
|
||||
|
||||
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
||||
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
||||
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
|
||||
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
||||
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
||||
|
||||
**A blank password must not set an empty password.** `set_password("")` would
|
||||
make the account log in with no password, so a validation policy
|
||||
(`default-user-settings-no-empty-password`, bound to the prompt via
|
||||
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
||||
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
||||
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
||||
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
||||
invalid).
|
||||
|
||||
## Outpost claim cache (`proxysession`) — 403 after a group add
|
||||
|
||||
The embedded outpost does not re-evaluate group membership per request. Each
|
||||
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
|
||||
(the inventory table carries both bare origins and trailing slashes;
|
||||
routing matches on hostname, verified 2026-09-02).
|
||||
`internal_host = http://caddy:8017`, all three flows set, and an
|
||||
Application attached — the two provisioning rules above.
|
||||
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
|
||||
2. **Route flip in ONE commit/deploy**: the vhost becomes
|
||||
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
|
||||
inside the `http://:8017` listener add `@<svc> host <vhost>` +
|
||||
@@ -521,22 +345,9 @@ EOF
|
||||
is no `authentik.policies.group` module in this build.
|
||||
*(unverified 2026-10-18: these module paths were not re-executed — importing
|
||||
outside a configured Django shell fails with `ImproperlyConfigured`)*
|
||||
- For live DB truth prefer the postgres container, and it works without any
|
||||
credential — the local socket is trusted, so a read-only audit is just
|
||||
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
||||
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
||||
before guessing: 2026.x renamed both tables and columns, and the models are
|
||||
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
||||
three `*_flow_id` columns; the proxy table is
|
||||
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
||||
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
||||
table in 2026.8), the LDAP table is
|
||||
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
|
||||
`authentik_core_application` attaches to a provider through `provider_id`
|
||||
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
||||
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
||||
is the same story from the ORM side: `type` lives in the subclass tables.
|
||||
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
||||
- The live database's table and column names (the 2026.x renames, the MTI parent
|
||||
keys, the read-only `psql` audit recipe) are reference lookup:
|
||||
[[Authentik Outpost Reference]].
|
||||
|
||||
## User provisioning playbook (user-level account)
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
|
||||
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
|
||||
Basic client auth, form-encoded token body, and the fork's userinfo-claims
|
||||
bug that leaves identity at the `sub` hash only) is in
|
||||
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
||||
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
||||
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
|
||||
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
|
||||
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Homepage Status Probes
|
||||
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
|
||||
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
|
||||
tags: [homepage, status, probes, e2e, homepage-status]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
|
||||
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
|
||||
tags:
|
||||
- infrastructure
|
||||
- homepage
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Observability Provisioning
|
||||
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
|
||||
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
|
||||
tags: [observability, prometheus, grafana, provisioning, deployment]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "Observability on Dankclaw"
|
||||
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
||||
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
||||
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
|
||||
timestamp: 2026-09-11
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
|
||||
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
|
||||
tags:
|
||||
- infrastructure
|
||||
- prometheus
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
- postgres
|
||||
timestamp: 2026-08-28
|
||||
---
|
||||
|
||||
# authentik-postgres
|
||||
|
||||
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
|
||||
OIDC provider. It is the only stateful store in the stack (users, flows,
|
||||
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
|
||||
now live here.
|
||||
|
||||
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
|
||||
- **Image:** `docker.io/library/postgres:16-alpine`
|
||||
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
|
||||
- **Health:** `pg_isready -d authentik -U authentik`
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
|
||||
(`pg_dump`), and failure modes.
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-09-16
|
||||
---
|
||||
|
||||
# authentik-server
|
||||
|
||||
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
||||
This is the service the Caddy `forward_auth` vhosts proxy to
|
||||
(`authentik-server:9000`). It also serves the login page at
|
||||
`https://auth.franksdank.cloud`.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
|
||||
- **Port:** `9000` (published; the outpost + admin + login all live here)
|
||||
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
|
||||
- **Health:** `http://localhost:9000/-/health/`
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
|
||||
forward_auth pattern, first-time setup, and failure modes.
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The Authentik worker — background tasks: event processing, provider"
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-09-16
|
||||
---
|
||||
|
||||
# authentik-worker
|
||||
|
||||
The Authentik worker — background tasks: event processing, provider
|
||||
synchronization, scheduled flows, and certificate renewal. It runs the same
|
||||
image as the server; the `command: worker` selects the role.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
|
||||
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
|
||||
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
|
||||
- **No port** — the worker has no HTTP endpoint.
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
|
||||
modes.
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Dankserver System Specs
|
||||
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
|
||||
description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
|
||||
tags:
|
||||
- dankserver
|
||||
- hardware
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
type: reference
|
||||
title: Dual R9700 Cooperation
|
||||
description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
|
||||
description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
|
||||
tags:
|
||||
- frank-medium
|
||||
- rocm
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
type: reference
|
||||
title: FastFlowLM NPU Inference
|
||||
description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
|
||||
description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
|
||||
tags:
|
||||
- fastflowlm
|
||||
- npu
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: FreeToken Builds
|
||||
description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
|
||||
description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
|
||||
tags: [freetoken, build, dankclaw, vllm, mtp]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: GGUF Compatibility and Runtime Lock-in
|
||||
description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
|
||||
description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
|
||||
tags: [gguf, llama.cpp, forks, quantization, speculative]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Heterogeneous Split Triage
|
||||
description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
|
||||
description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
|
||||
tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: LLM Bench Windows and Measurement
|
||||
description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
|
||||
description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
|
||||
tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: LLM Performance Diagnosis
|
||||
description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
|
||||
description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
|
||||
tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Llama Cpp Multi-Device Splits
|
||||
description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
|
||||
description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
|
||||
tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Llama Cpp Serving Sizing
|
||||
description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
|
||||
description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
|
||||
tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Model Swap and Rollback
|
||||
description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
|
||||
description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
|
||||
tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: PCIe Link and Atomics
|
||||
description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
|
||||
description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
|
||||
tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
---
|
||||
type: reference
|
||||
title: Ethernet Bandwidth Math
|
||||
description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
|
||||
description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- research
|
||||
timestamp: 2026-08-17
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
|
||||
@@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2 [fp16 = 2 bytes]
|
||||
- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
|
||||
- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
|
||||
|
||||
**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
|
||||
**Worked examples (1 boundary = 2 PP stages):**
|
||||
```
|
||||
1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
|
||||
```
|
||||
|
||||
**Example — same model, batch=64:**
|
||||
```
|
||||
1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
|
||||
```
|
||||
|
||||
**Example — 36B model, 2 PP stages, batch=1:**
|
||||
```
|
||||
1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
|
||||
27B, batch=1: 1 boundary × 4096 × 2 bytes = 8,192 bytes/tok = 8 KB
|
||||
27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288 = 512 KB/tok
|
||||
36B, batch=1: 1 boundary × 6144 × 2 bytes = 12,288 bytes/tok = 12 KB
|
||||
```
|
||||
|
||||
### 1.2 Tensor Parallelism (TP) — for comparison
|
||||
@@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2 [L all-reduces total
|
||||
| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
|
||||
| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
|
||||
|
||||
### 2.3 Feasibility matrix
|
||||
### 2.3 Feasibility
|
||||
|
||||
Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
|
||||
|
||||
**27B model — PP (1 boundary), batch=1:**
|
||||
|
||||
| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
|
||||
| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
|
||||
| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
|
||||
|
||||
**27B model — PP (1 boundary), batch=16:**
|
||||
|
||||
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
|
||||
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
|
||||
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
|
||||
|
||||
**27B model — PP (1 boundary), batch=64:**
|
||||
|
||||
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
|
||||
| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
|
||||
| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
|
||||
|
||||
**27B model — TP (32 layers), batch=1:**
|
||||
|
||||
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
|
||||
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
|
||||
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
|
||||
|
||||
**27B model — TP (32 layers), batch=16:**
|
||||
|
||||
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
|
||||
| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
|
||||
| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
|
||||
|
||||
**27B model — TP (32 layers), batch=64:**
|
||||
|
||||
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||
|---|---|---|---|
|
||||
| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
|
||||
| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
|
||||
| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
|
||||
Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
|
||||
**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
|
||||
feasible on every link class — even the heaviest, 27B at batch=64 and 100
|
||||
tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
|
||||
except batch=64:
|
||||
|
||||
**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
|
||||
|
||||
@@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
|
||||
**Petals [1]:**
|
||||
- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
|
||||
- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
|
||||
- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
|
||||
- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
|
||||
- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
|
||||
- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)
|
||||
|
||||
**llama.cpp RPC [2]:**
|
||||
- Model weights and KV cache distributed across nodes proportionally to memory
|
||||
@@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute
|
||||
|
||||
## Appendix: Full bandwidth formulae
|
||||
|
||||
### Pipeline Parallelism (generation, batch=1)
|
||||
### Pipeline and Tensor Parallelism (generation)
|
||||
|
||||
`N` is the batch size; `N = 1` is the batch=1 form.
|
||||
|
||||
```
|
||||
bytes_per_boundary_per_tok = hidden_size × 2 [fp16]
|
||||
total_bytes_per_tok = num_boundaries × hidden_size × 2
|
||||
bytes_per_boundary_per_tok = hidden_size × N × 2 [fp16, PP boundary]
|
||||
bytes_per_layer_allreduce = hidden_size × N × 2 [TP, one all-reduce per layer]
|
||||
total_bytes_per_tok = num_boundaries × hidden_size × N × 2 [PP]
|
||||
total_bytes_per_tok = num_layers × hidden_size × N × 2 [TP]
|
||||
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
|
||||
```
|
||||
|
||||
### Pipeline Parallelism (generation, batch=N)
|
||||
|
||||
```
|
||||
bytes_per_boundary_per_tok = hidden_size × N × 2
|
||||
total_bytes_per_tok = num_boundaries × hidden_size × N × 2
|
||||
```
|
||||
|
||||
### Tensor Parallelism (generation, batch=1)
|
||||
|
||||
```
|
||||
bytes_per_layer_allreduce = hidden_size × 2
|
||||
total_bytes_per_tok = num_layers × hidden_size × 2
|
||||
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
|
||||
```
|
||||
|
||||
### Tensor Parallelism (generation, batch=N)
|
||||
|
||||
```
|
||||
total_bytes_per_tok = num_layers × hidden_size × N × 2
|
||||
```
|
||||
|
||||
### With 8-bit quantization (like Petals)
|
||||
|
||||
```
|
||||
# Simply divide all fp16 byte counts by 2
|
||||
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
|
||||
# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
|
||||
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
|
||||
```
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Expected Performance
|
||||
description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
|
||||
description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- performance
|
||||
@@ -13,7 +13,7 @@ timestamp: 2026-08-14
|
||||
|
||||
This page estimates the speed of a 25-40B model split across two machines
|
||||
over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
|
||||
and the measured numbers in [[Mixed-Vendor Experiments]].
|
||||
and the mixed-vendor measurements in [[Multi-Node Runtimes]].
|
||||
|
||||
## The Bandwidth Ceiling Is Not the Limit
|
||||
|
||||
@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
|
||||
## See Also
|
||||
|
||||
- [[Ethernet Bandwidth Math]]
|
||||
- [[Mixed-Vendor Experiments]]
|
||||
- [[Multi-Node Runtimes]]
|
||||
- [[Sample Script Set]]
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
title: "Mixed-Vendor Experiments"
|
||||
description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- research
|
||||
timestamp: 2026-08-14
|
||||
---
|
||||
@@ -0,0 +1,257 @@
|
||||
---
|
||||
type: reference
|
||||
title: Multi-Node Runtimes
|
||||
description: "How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- llamacpp
|
||||
- vllm
|
||||
- speculative
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Multi-Node Runtimes
|
||||
|
||||
Two runtimes can put one model on two machines: llama.cpp over the ggml-rpc
|
||||
backend, and vLLM with pipeline parallelism. This page owns the cross-runtime
|
||||
facts — how each is built and launched, what each can and cannot be combined
|
||||
with, and the mixed-vendor verdicts. Facts owned by a sibling page are linked,
|
||||
not restated. Bytes-per-token arithmetic and link budgets belong to
|
||||
[[Ethernet Bandwidth Math]], throughput estimates to [[Expected Performance]],
|
||||
the real deployment and its verified gotchas to [[Qwen3.8 PP Deployment]], the
|
||||
educational scripts to [[Sample Script Set]], drafter mechanisms to
|
||||
[[DFlash DSpark Speculative Decoding]], single-host multi-GPU placement to
|
||||
[[Llama Cpp Multi-Device Splits]].
|
||||
|
||||
## What each runtime is
|
||||
|
||||
Both distribute by **layers**, not inside layers.
|
||||
|
||||
| | llama.cpp over ggml-rpc | vLLM pipeline parallelism |
|
||||
| --- | --- | --- |
|
||||
| unit of distribution | a contiguous slice of layers (`--split-mode layer`, the default) | a contiguous slice of layers (`--pipeline-parallel-size`) |
|
||||
| process model | one `llama-server`/`llama-cli` on the head; a `ggml-rpc-server` per worker | one `vllm serve` per node, one engine across them |
|
||||
| what crosses the wire | whatever graph work the head offloads to the remote devices | the hidden state at each stage boundary, point to point |
|
||||
| KV cache | distributed with the weights; the KV of a layer lives with the owner of that layer | each stage holds the KV of the layers it owns |
|
||||
| speculation together with the split | yes in llama.cpp (see below) | no, per vLLM's own docs |
|
||||
|
||||
Upstream's own summary is why layer-wise is the ethernet choice: pipeline
|
||||
parallel "minimizes data transfers between GPUs but requires many tokens to
|
||||
scale well", while tensor parallel "does multiple cross-GPU reductions per
|
||||
layer … much more bottlenecked by the GPU interconnect speed".
|
||||
|
||||
Do not carry the tensor path across machines: `--split-mode tensor` is
|
||||
experimental, needs flash attention, rejects quantized KV, and is not
|
||||
implemented for the MoE and hybrid architectures the guide lists — and
|
||||
per-tensor placement across vendors is not expressible (§1 of [[Llama Cpp Multi-Device Splits]]).
|
||||
|
||||
## llama.cpp over ggml-rpc
|
||||
|
||||
### Build and launch
|
||||
|
||||
Both sides take `-DGGML_RPC=ON` next to that host's own backend:
|
||||
|
||||
```bash
|
||||
cmake .. -DGGML_CUDA=ON -DGGML_RPC=ON # a CUDA worker; -DGGML_HIP=ON for an AMD one
|
||||
cmake --build . --config Release
|
||||
```
|
||||
|
||||
Worker side — `ggml-rpc-server` exposes ggml devices on the remote host:
|
||||
|
||||
```bash
|
||||
bin/ggml-rpc-server # endpoint 127.0.0.1:50052 by default
|
||||
bin/ggml-rpc-server --device CUDA0 -c # or CUDA_VISIBLE_DEVICES=0; -c = local cache
|
||||
```
|
||||
|
||||
- It exposes every accelerator it can see; a host with no accelerator exposes a
|
||||
single `CPU` device. `--device` and `CUDA_VISIBLE_DEVICES` are equivalent
|
||||
ways to narrow that set.
|
||||
- `-c` keeps large tensors in `$HOME/.cache/llama.cpp/rpc` (redirect with
|
||||
`LLAMA_CACHE`) so a big model does not re-stream per load; that cache
|
||||
filling the worker disk is a recorded fleet incident ([[Qwen3.8 PP Deployment]]).
|
||||
|
||||
Head side — build the local backends plus `-DGGML_RPC=ON`, then name the
|
||||
workers:
|
||||
|
||||
```bash
|
||||
llama-server -m model.gguf --rpc 192.168.88.10:50052,192.168.88.11:50052 -ngl 99
|
||||
```
|
||||
|
||||
`--rpc` is a comma-separated list of `host:port`, one entry per
|
||||
`ggml-rpc-server`; the head's own devices stay in the pool next to the remote
|
||||
ones. Launch scripts for this shape: [[Sample Script Set]].
|
||||
|
||||
### Split, context, KV
|
||||
|
||||
- Default placement: weights **and** KV across all devices, local and remote,
|
||||
"in proportion to each device's available memory". Override with
|
||||
`--tensor-split`, whose proportions follow the order in `--device`. Why the
|
||||
fleet always pins it (a Strix reports a huge GTT window, so the automatic
|
||||
split lands on the slow device) is [[Qwen3.8 PP Deployment]]'s.
|
||||
- `-sm layer` (the default) is pipeline parallelism: each device holds a
|
||||
contiguous slice of layers and the KV cache of a layer lives on the device
|
||||
that owns that layer.
|
||||
- KV types are the usual `-ctk` / `-ctv`, `f16` by default. `--fit` is on by
|
||||
default and sizes unset arguments to device memory; turn it off to hold a
|
||||
context you set on purpose. Context is one global `--ctx-size` — the RPC
|
||||
README documents no per-node context or per-node KV knob.
|
||||
- Upstream's out-of-memory ladder (stated for tensor mode, useful order
|
||||
anyway): lower `--ctx-size`, then `--parallel`, then `--n-gpu-layers`.
|
||||
- Transport is TCP; RDMA is negotiated at the handshake with no flag changes
|
||||
and falls back to TCP unless both peers can use it (Linux RoCEv2 via
|
||||
`libibverbs`, macOS RDMA over Thunderbolt 5 via `librdma`). It is point to
|
||||
point — name the peer's RDMA-capable address in `--rpc` or the session stays
|
||||
on TCP. `GGML_RPC_NO_RDMA=1` forces TCP; `GGML_RPC_DEBUG` traces the backend.
|
||||
|
||||
Upstream calls the backend a proof of concept, "fragile and insecure" (see the
|
||||
Security section). Two limits shape the design: decode is bound by the per-step
|
||||
round trip, not by the link's bytes ([[Ethernet Bandwidth Math]]), and every
|
||||
node must run the identical build or the handshake hangs ([[Qwen3.8 PP Deployment]]).
|
||||
|
||||
## vLLM pipeline parallelism
|
||||
|
||||
> **Provenance.** Flags and mechanics below were recorded 2026-08-14 and are
|
||||
> not re-checked upstream in this pass (see "Unverified in this pass"); the
|
||||
> incompatibility lines, the method catalog and the issue history are re-verified.
|
||||
|
||||
- `--pipeline-parallel-size` divides the layers into contiguous stages;
|
||||
`--tensor-parallel-size` still works inside a node, so the canonical two-node
|
||||
shape is tp = GPUs per node, pp = nodes.
|
||||
- Two backends: Ray (`--distributed-executor-backend ray`, the worker joining
|
||||
a `ray start --head`) or multiprocessing (`--distributed-executor-backend mp`
|
||||
with `--nnodes`, `--node-rank`, `--master-addr`, `--headless` on the
|
||||
follower). `mp` avoids the Ray dependency; Ray brings the cluster manager.
|
||||
- What the interconnect must carry: one activation tensor per stage boundary
|
||||
per forward pass, point to point — not a per-layer reduction. That is the
|
||||
whole cross-node traffic story; the byte arithmetic and link budgets are
|
||||
[[Ethernet Bandwidth Math]]. Pin the NIC vLLM talks on
|
||||
(`NCCL_SOCKET_IFNAME`) rather than letting it guess an interface.
|
||||
- The PP bubble: with N stages and one sequence in flight, one stage works
|
||||
while N-1 idle, so pp=2 costs half the machine at batch one. Batching is the
|
||||
cure — pipeline parallelism "requires many tokens to scale well" — and the
|
||||
microbatch formula is [[Ethernet Bandwidth Math]] §5.3.
|
||||
- KV and context: each stage keeps the KV of its own layers, so the pool grows
|
||||
with the stage count, and `--max-model-len` is not reduced by the split
|
||||
(recorded 2026-08-14, unverified here).
|
||||
- What it cannot combine with, in vLLM's words: "Pipeline parallelism is not
|
||||
composable with speculative decoding as of `vllm<=0.15.0`", and "Speculative
|
||||
decoding with a draft models is not supported in `vllm<=0.10.0`". Composing
|
||||
the two has been requested since 2024 and closed stale (#6911, #10615); PP
|
||||
also carries a Qwen3.5 bug report closed stale without a fix (#36643).
|
||||
- Method catalog at v0.25.1: EAGLE, MTP, draft model, PARD, MLP, n-gram, suffix
|
||||
decoding, hidden-state extraction, a custom proposer class, dynamic
|
||||
speculative decoding; `dflash` is an accepted `method` value.
|
||||
|
||||
## Speculative decoding across the split
|
||||
|
||||
llama.cpp chains several implementations behind one comma-separated
|
||||
`--spec-type` list; the split does not change the chain, only where its passes
|
||||
run. Mechanisms, chaining laws and drafter KV sizing are
|
||||
[[DFlash DSpark Speculative Decoding]]'s; only the split interaction is here.
|
||||
|
||||
| Method | llama.cpp type | Over ggml-rpc | With vLLM PP | Why |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| MTP | `draft-mtp` | works; not documented upstream, but run this way on this fleet's split head with acceptance measured ([[Qwen3.8 PP Deployment]]) | no | Context-backed drafter: it needs a draft context and runs in the head process, so RPC devices are simply more devices under it |
|
||||
| DFlash | `draft-dflash` | same gate, same status as MTP | no | Same draft-context gate (`ctx_dft`); the drafter's KV lands on whichever device the draft context is built on — the one thing to check on a split |
|
||||
| ngram-mod | `ngram-mod` | safest of the four: no draft model, no draft context, no remote draft pass | no | Draftless — proposals come from cached token n-grams in the head, so the split adds no device work |
|
||||
| DSpark | `draft-dspark` | **unproven over RPC** — not a missing feature | no | `draft-dspark` exists upstream; nobody has measured it with the target split across two machines |
|
||||
|
||||
**Correction (2026-10-19).** A retired stub in this directory asserted "DSpark
|
||||
is vLLM-only." False. `draft-dspark` is in upstream llama.cpp — parsed in
|
||||
`common/arg.cpp`, mapped to `COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK` in
|
||||
`common/speculative.cpp`, where it carries a Markov head and an anchor-first
|
||||
block layout and aborts with `DSpark draft has no confidence head: please set
|
||||
--spec-draft-p-min 0` when the drafter has no confidence head. What is unproven
|
||||
is running it over RPC, and that is the honest limit of the claim.
|
||||
|
||||
The vLLM column is uniform because that statement is blanket: it names pipeline
|
||||
parallelism, not a method, so n-gram and suffix are excluded as written. vLLM
|
||||
gives no mechanism with it; "the drafter must sit next to the target's layers,
|
||||
which a stage split separates" is a reading, not upstream's words.
|
||||
|
||||
Upstream likewise documents neither for nor against `--rpc` plus
|
||||
`--spec-type`. What the implementation does say is structural: every
|
||||
speculative implementation is built in the head process against the head's
|
||||
device set, and draftless types need no draft context at all. Routing draft
|
||||
passes to a remote worker, however, has never been tested.
|
||||
|
||||
## Mixed-vendor reality
|
||||
|
||||
What works today:
|
||||
|
||||
- **One vendor per node.** RPC is backend-agnostic by construction: upstream's
|
||||
own diagram has a host exposing CUDA0+CUDA1, a host exposing Metal, and a
|
||||
host exposing CUDA0+CPU, all driven by one main host over TCP. Each node
|
||||
builds its own backend plus `-DGGML_RPC=ON`; nothing speaks CUDA-to-ROCm.
|
||||
- **Identical build tag on every node**, worker restarted before head. The
|
||||
verified version-skew gotcha is [[Qwen3.8 PP Deployment]]'s.
|
||||
- **Speculation on the head** while the split carries the target (table above).
|
||||
- **RDMA when both peers have it**, auto-negotiated at the handshake; the size
|
||||
of the gain is third-party and unverified here.
|
||||
|
||||
Unproven or broken:
|
||||
|
||||
- **One binary holding both CUDA and ROCm for a same-host layer split**:
|
||||
reported to compile and then list only CUDA devices (issue #11506,
|
||||
third-party, unverified). Vulkan is the reported single-binary cross-vendor
|
||||
path, and single-host placement is [[Llama Cpp Multi-Device Splits]]'s, whose
|
||||
verdict is that per-tensor placement across a vendor mix never works.
|
||||
- **vLLM across a CUDA node and a ROCm node**: no known experiment; both ends
|
||||
run the same backend (recorded 2026-08-14). AMD's own two-Halo clustering
|
||||
guide is tensor parallelism over RCCL, not a mixed-vendor split.
|
||||
- **DSpark over RPC**: unproven, see the correction above.
|
||||
- **The exact NVIDIA-desktop + Strix-Halo-NUC pair has no published
|
||||
benchmark**, and the third-party mixed-vendor figures it is estimated from
|
||||
(Metal+CUDA over 10 GbE, two Strix Halos over USB4, a three-card CUDA+ROCm set
|
||||
over 1 GbE, a Vulkan triple in one box, an AMD four-node Strix cluster) live on
|
||||
[[Expected Performance]] and [[Ethernet Bandwidth Math]], each named with its
|
||||
model and hardware. None was re-verified in this pass.
|
||||
|
||||
## Security
|
||||
|
||||
`ggml-rpc-server` has **no authentication and no encryption**. Upstream's own
|
||||
warning: the backend is "in a proof-of-concept development stage. As such, the
|
||||
functionality is fragile and insecure. Never run the RPC server on an open
|
||||
network or in a sensitive environment!" It binds `127.0.0.1:50052` by default —
|
||||
safe by accident. Bind a routable address and any host that can reach the port
|
||||
can allocate memory there and supply the tensors it computes on. Trusted LAN
|
||||
only, firewalled or tunnelled; never a public address.
|
||||
|
||||
## Unverified in this pass
|
||||
|
||||
Every item was first recorded 2026-08-14, in the research that preceded this
|
||||
page. Nothing is silently deleted; none is backed by the sources cited below.
|
||||
|
||||
- **RPC protocol version and handshake rule** — recorded as protocol `5.0.0`
|
||||
with "major must match, server minor must be ≥ client". The README's own
|
||||
banner prints `Starting RPC server v3.0.0`; the conflict is unresolved.
|
||||
- **RPC internals** — a hard cap of 16 servers, synchronous graph compute, no
|
||||
tensor copy between two RPC servers, workers loading tensors on first use, a
|
||||
hash-keyed cache above roughly 10 MB, asserts on quantized tensors whose rows
|
||||
are not 512-aligned, and `-rpc` on `llama-bench` (the README names only
|
||||
`llama-cli` and `llama-server`).
|
||||
- **vLLM flag surface** — `--pipeline-parallel-size`, `--tensor-parallel-size`,
|
||||
`--distributed-executor-backend {ray,mp}`, `--nnodes`, `--node-rank`,
|
||||
`--master-addr`, `--headless`, the `ray start --head` join,
|
||||
`NCCL_SOCKET_IFNAME`.
|
||||
- **vLLM mechanics** — one activation handoff per stage boundary per forward
|
||||
pass, a worked ~512 MB handoff for a hidden-5120 model at batch 8 and 2048
|
||||
positions, `--max-model-len` unchanged by PP, ~50% idle at pp=2 batch one.
|
||||
- **vLLM throughput verdicts** for a 27B across two nodes over gigabit-class
|
||||
links; superseded as an estimate by [[Expected Performance]].
|
||||
- **Mixed-vendor claims** — a CUDA+HIP build listing only CUDA devices
|
||||
(#11506), Vulkan as the universal cross-vendor backend, AMD's two-Halo
|
||||
clustering guide being RCCL tensor parallelism, WiFi costing about ten times
|
||||
wired, the slowest hop setting the decode pace, RDMA worth a few percent.
|
||||
- **Draft over RPC** — that the draft model stays on the head, and that routing
|
||||
draft passes to a remote worker has never been confirmed.
|
||||
|
||||
## Citations
|
||||
|
||||
- [llama.cpp RPC README](https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md), accessed 2026-10-19 — server usage and device exposure, `-DGGML_RPC=ON` on both sides, the `--rpc host:port` list, proportional weight+KV split and `--tensor-split`, the `-c` local cache, RDMA negotiation and `GGML_RPC_NO_RDMA`, the proof-of-concept and no-authentication warning.
|
||||
- [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md), accessed 2026-10-19 — split modes (`-sm layer` = pipeline parallelism, KV with the layer owner), `-ts`/`-ngl`/`-ctk`/`-ctv`/`-fit`, the pipeline-versus-tensor summary, the tensor-mode limits.
|
||||
- [vLLM speculative decoding docs, v0.25.1](https://docs.vllm.ai/en/v0.25.1/features/speculative_decoding/), accessed 2026-10-19 — the pipeline-parallelism incompatibility statement, the draft-model version floor, the method catalog.
|
||||
- [llama.cpp `common/speculative.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/common/speculative.cpp), accessed 2026-10-19 — the speculative implementation set including `draft-dspark`, its draft-context gate, its confidence-head error.
|
||||
- vLLM issue #6911, "[Feature]: Combine pipeline parallelism with speculative decoding", opened 2024-07-29, closed not-planned 2025-03-28, accessed 2026-10-19.
|
||||
- vLLM issue #10615, "[Usage]: Does speculative decoding support pipeline parallelism?", opened 2024-11-25, closed not-planned 2025-04-18, accessed 2026-10-19.
|
||||
- vLLM issue #36643, "[Bug]: Qwen3.5 does not work with pipeline parallelism", opened 2026-03-10, closed not-planned 2026-09-24, accessed 2026-10-19.
|
||||
@@ -1,9 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
title: "Qwen3.6 27B Spec Methods"
|
||||
description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- research
|
||||
timestamp: 2026-08-14
|
||||
---
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
|
||||
description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
|
||||
tags:
|
||||
- llm
|
||||
- pipeline
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Sample Script Set
|
||||
description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
|
||||
description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- sample
|
||||
@@ -255,7 +255,7 @@ is 30 GiB.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[llama.cpp RPC]]
|
||||
- [[Multi-Node Runtimes]]
|
||||
- [[Expected Performance]]
|
||||
- [[Ethernet Bandwidth Math]]
|
||||
- [[Qwen3.6 27B Spec Methods]]
|
||||
- [[DFlash DSpark Speculative Decoding]]
|
||||
@@ -28,28 +28,24 @@ graph LR
|
||||
end
|
||||
```
|
||||
|
||||
- [[llama.cpp RPC]] — the working, production-capable path for two
|
||||
heterogeneous nodes over ethernet.
|
||||
- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
|
||||
multiprocessing and does not compose with speculative decoding yet.
|
||||
|
||||
## What the Research Found
|
||||
|
||||
- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
|
||||
- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
|
||||
(about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
|
||||
tokens per second on paper; the real limit is latency and RPC overhead.
|
||||
- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
|
||||
works today via llama.cpp RPC with separate CUDA and ROCm builds. The
|
||||
exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
|
||||
published numbers yet.
|
||||
- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
|
||||
methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
|
||||
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
|
||||
- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
|
||||
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
|
||||
|
||||
## Deployment
|
||||
|
||||
- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
|
||||
- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
|
||||
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
|
||||
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
|
||||
`--rpc`, default split is proportional to reported free memory (pin 1,1),
|
||||
@@ -57,7 +53,7 @@ graph LR
|
||||
|
||||
## Sample Scripts
|
||||
|
||||
- [[Sample Script Set]] — a two-node llama.cpp RPC split of
|
||||
- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
|
||||
Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
|
||||
the largest context that fits. For educational reading only. Do not run
|
||||
it in production.
|
||||
@@ -68,3 +64,9 @@ graph LR
|
||||
- [[frank-zealot]]
|
||||
- [[Dankserver System Specs]]
|
||||
- [[Dankclaw System Specs]]
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | Owns |
|
||||
|---|---|
|
||||
| [[Multi-Node Runtimes]] | How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts. |
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
title: "llama.cpp RPC"
|
||||
description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- research
|
||||
timestamp: 2026-08-14
|
||||
---
|
||||
@@ -1,9 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
title: "vLLM Pipeline Parallelism"
|
||||
description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
|
||||
tags:
|
||||
- pipeline-parallelism
|
||||
- research
|
||||
timestamp: 2026-08-14
|
||||
---
|
||||
@@ -1,3 +1,13 @@
|
||||
---
|
||||
type: "howto"
|
||||
title: "Quantization Production"
|
||||
description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
|
||||
tags:
|
||||
- llm
|
||||
- quantization
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Quantization Production — making our own quants, per card and runtime
|
||||
|
||||
**Research page (2026-10-08).** Question: how do WE produce quantizations
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: RDNA4 Container Serving
|
||||
description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
|
||||
description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
|
||||
tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: RDNA4 MoE Expert Offload VRAM Budget
|
||||
description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
|
||||
description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
|
||||
tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
|
||||
description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
|
||||
tags:
|
||||
- llm
|
||||
timestamp: 2026-09-03
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
|
||||
description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
|
||||
tags:
|
||||
- llm
|
||||
timestamp: 2026-09-03
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Tcclaviger Fork Ops
|
||||
description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
|
||||
description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
|
||||
tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
|
||||
description: "Locator for every vLLM page in this vault."
|
||||
tags:
|
||||
- llm
|
||||
- vllm
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
type: reference
|
||||
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
|
||||
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
|
||||
description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
|
||||
tags:
|
||||
- dankclaw
|
||||
- iommu
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
type: reference
|
||||
title: Frontier MoE on Dankserver — what fits, on which runtime
|
||||
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
|
||||
description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
|
||||
tags:
|
||||
- dankserver
|
||||
- r9700
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
|
||||
description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot.
|
||||
description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
|
||||
tags:
|
||||
- llamacpp
|
||||
- cuda
|
||||
@@ -9,6 +9,7 @@ tags:
|
||||
- moe
|
||||
- ornith
|
||||
- mtp
|
||||
- retired-endpoint
|
||||
timestamp: 2026-08-24
|
||||
---
|
||||
|
||||
@@ -123,4 +124,4 @@ llama-server \
|
||||
|
||||
- [[deployment]] — deploy and update process
|
||||
- [[Dankserver System Specs]] — the box
|
||||
- [[frank-redo-2026-08-17]] — fleet topology history
|
||||
- [[frank-redo-2026-08-17]] — fleet topology histor
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
type: guide
|
||||
type: howto
|
||||
title: LLM Deployment
|
||||
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
|
||||
tags:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
|
||||
description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
|
||||
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
|
||||
tags:
|
||||
- llm
|
||||
- frank-archon
|
||||
@@ -11,6 +11,7 @@ tags:
|
||||
- r9700
|
||||
- tp2
|
||||
- expert-offload
|
||||
- endpoint
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
@@ -139,8 +140,10 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
|
||||
| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
|
||||
| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
|
||||
|
||||
Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
|
||||
(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
|
||||
Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
|
||||
pages are **not** superseded here: [[frank-medium]] is the locator for that
|
||||
endpoint's whole shape history, including the unresolved end date of the
|
||||
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
|
||||
|
||||
## Laws learned here
|
||||
|
||||
@@ -438,3 +441,4 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
|
||||
[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
|
||||
[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
|
||||
[[frank-medium-radiance-mxfp4]], [[LLM/index]].
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
||||
description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
|
||||
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
|
||||
tags:
|
||||
- llm
|
||||
- frank-dragoon
|
||||
@@ -9,6 +9,7 @@ tags:
|
||||
- strix-halo
|
||||
- gfx1151
|
||||
- halogen
|
||||
- endpoint
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
@@ -234,3 +235,4 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
|
||||
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
|
||||
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
|
||||
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
|
||||
---
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
timestamp: 2026-09-17
|
||||
type: reference
|
||||
title: frank-medium — Batch and Ubatch Sweep
|
||||
description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides.
|
||||
description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
|
||||
tags:
|
||||
- benchmark
|
||||
- llamacpp
|
||||
@@ -11,6 +11,7 @@ tags:
|
||||
- dankserver
|
||||
- layer-split
|
||||
---
|
||||
- retired-endpoint
|
||||
|
||||
# frank-medium — Batch and Ubatch Sweep
|
||||
|
||||
@@ -222,4 +223,5 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
|
||||
difference: with no KV to attend over, per-pass overhead dominates and a
|
||||
bigger micro-batch wins; at 64k of resident KV it does not.
|
||||
- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
|
||||
- [[Dankserver System Specs]] — hardware inventory
|
||||
- [[Dankserver System Specs]] — hardware inventory---
|
||||
timestamp: 2026-09-17
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
|
||||
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
|
||||
description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -12,6 +12,7 @@ tags:
|
||||
- gfx1201
|
||||
- r9700
|
||||
- layer-split
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-20
|
||||
---
|
||||
|
||||
@@ -730,3 +731,4 @@ fleet.
|
||||
- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
|
||||
geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
|
||||
why the logical batch is free, and the harness traps
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: runbook
|
||||
type: howto
|
||||
title: frank-medium Model Swap
|
||||
description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
|
||||
description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path."
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -9,6 +9,7 @@ tags:
|
||||
- vllm-radiance
|
||||
- huggingface
|
||||
- runbook
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-20
|
||||
---
|
||||
|
||||
@@ -155,9 +156,10 @@ back:
|
||||
2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
|
||||
3. Re-run the capability battery.
|
||||
|
||||
The 2026-09-20 rollback target is the MXFP4 checkpoint at
|
||||
`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single
|
||||
safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
|
||||
The 2026-09-20 rollback target is the MXFP4 checkpoint with
|
||||
`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that
|
||||
payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a
|
||||
local file.
|
||||
|
||||
## 2026-09-20 swap record (MXFP4 → FP8)
|
||||
|
||||
@@ -184,3 +186,4 @@ safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
|
||||
DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
|
||||
smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
|
||||
vs 19.4 GiB).
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: frank-medium vllm-radiance MXFP6 Runbook
|
||||
description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
|
||||
description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -14,6 +14,7 @@ tags:
|
||||
- rccl
|
||||
- pcie-atomics
|
||||
- quadlet
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-21
|
||||
---
|
||||
|
||||
@@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
|
||||
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
|
||||
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
|
||||
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
|
||||
drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
|
||||
decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
|
||||
`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
|
||||
back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
|
||||
vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
|
||||
drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
|
||||
below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
|
||||
rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
|
||||
`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
|
||||
rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
|
||||
`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
|
||||
image back to `localhost/radiance-vllm:0.9.3` (re-pull
|
||||
`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
|
||||
486,464 tokens with bf16 KV and held the full 262,144 window).
|
||||
|
||||
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
|
||||
@@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
|
||||
with growing history) shows what happens at that limit: turns alternate between
|
||||
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
|
||||
and must be prefilled again, because two families per card plus their cached
|
||||
prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
|
||||
both replicas steady — and per-family pinning kept the fast turns on the replica
|
||||
that holds the prefix.
|
||||
prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
|
||||
turns on the replica that holds the prefix.
|
||||
|
||||
Read that as a capacity budget, not a defect: two concurrent large-context
|
||||
sessions per card are comfortable, three or more of that size trade turns
|
||||
@@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here.
|
||||
the same image and kernel 7.0.0-30 failing identically.
|
||||
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
|
||||
this page compares against.
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: frank-medium ROCmFPX Runbook
|
||||
description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201).
|
||||
description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required."
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -10,6 +10,7 @@ tags:
|
||||
- mtp
|
||||
- gfx1201
|
||||
- r9700
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-15
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
type: reference
|
||||
title: frank-medium (retired) — the Dankserver endpoint before frank-archon
|
||||
description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree."
|
||||
timestamp: 2026-10-19
|
||||
tags:
|
||||
- llm
|
||||
- dankserver
|
||||
- frank-medium
|
||||
- retired
|
||||
- retired-endpoint
|
||||
---
|
||||
|
||||
# frank-medium (retired) — the Dankserver endpoint before frank-archon
|
||||
|
||||
`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live
|
||||
endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every
|
||||
shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon`
|
||||
name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** —
|
||||
which era page holds which payload's evidence — and carries no measurements: every
|
||||
number stays in the era page that measured it.
|
||||
|
||||
## Eras
|
||||
|
||||
Windows are quoted from each era page's own banner; no date here is derived.
|
||||
|
||||
| Window | Runtime | Payload | Shape | Page | Rollback |
|
||||
|---|---|---|---|---|---|
|
||||
| 2026-09-15 → 2026-09-16 | llama.cpp, ROCmFPX fork pinned at commit `36e9acd` | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) | single card; the fork is the only loader | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork |
|
||||
| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags |
|
||||
| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape |
|
||||
| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4, then MXFP6 (`paroquant_mxfp6`) with bf16 KV | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` |
|
||||
|
||||
`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run
|
||||
on the layer-split pair at its deployed shape, and its numbers belong to that
|
||||
campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of
|
||||
the 2026-09-20 swap procedure — while the procedure itself is owned by
|
||||
[[Model Swap and Rollback]].
|
||||
|
||||
## Laws
|
||||
|
||||
None restated. Each era page carries its own laws and links them to the technique
|
||||
page that owns the mechanism ([[PCIe Link and Atomics]],
|
||||
[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]],
|
||||
[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]).
|
||||
|
||||
## Open questions
|
||||
|
||||
Two date claims in the era pages do not reconcile, and both are left standing rather
|
||||
than silently corrected:
|
||||
|
||||
- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the
|
||||
next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The
|
||||
MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which
|
||||
one directly followed the ROCmFPX fork cannot be settled from the repo.
|
||||
- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while
|
||||
[[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded
|
||||
fortnight on that endpoint or a stale banner; nothing else in the repo covers
|
||||
2026-09-26 → 10-10 for port 7770.
|
||||
|
||||
## Files
|
||||
|
||||
No repo files own this endpoint: its units, env and serve scripts were renamed to
|
||||
`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds
|
||||
retired serve scripts from these eras as reference only.
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
|
||||
description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
|
||||
description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went."
|
||||
tags:
|
||||
- flm
|
||||
- fastflowlm
|
||||
@@ -10,6 +10,7 @@ tags:
|
||||
- dankclaw
|
||||
- frank-npu
|
||||
- retired
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-11
|
||||
---
|
||||
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This page records the full redo of the inference fleet on 2026-08-17."
|
||||
description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way."
|
||||
tags:
|
||||
- llm
|
||||
- frank
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-03
|
||||
---
|
||||
|
||||
@@ -223,3 +224,4 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17
|
||||
evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
|
||||
the main TTFT lever for pi, which re-sends the same system prompt + tool
|
||||
schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV"
|
||||
description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
|
||||
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
|
||||
tags:
|
||||
- llm
|
||||
- frank-zealot
|
||||
@@ -9,6 +9,7 @@ tags:
|
||||
- llama-cpp
|
||||
- gguf
|
||||
- rtx3090
|
||||
- endpoint
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
type: reference
|
||||
title: FreeToken Benchmark Sweep — Dankserver RTX 3090
|
||||
description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2.
|
||||
description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime."
|
||||
tags:
|
||||
- freetoken
|
||||
- benchmark
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
|
||||
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
|
||||
description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required."
|
||||
tags:
|
||||
- ikllama
|
||||
- llamacpp
|
||||
@@ -11,7 +11,8 @@ tags:
|
||||
- ornith
|
||||
- kv-cache
|
||||
- hadamard
|
||||
timestamp: 2026-08-26
|
||||
- retired-endpoint
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
|
||||
@@ -259,4 +260,4 @@ spec / `--spec-type` flags. The model file is not changed either way.
|
||||
build flags incl. `GGML_CUDA_FA_ALL_QUANTS` and `GGML_IQK_FA_ALL_QUANTS`
|
||||
- [ik_llama.cpp serverAppConfigs-docs/parameters.md](https://github.com/ikawrakow/ik_llama.cpp/blob/main/docs/parameters.md) —
|
||||
q8_KV (PR 208), K/V Hadamard options, `-cuda graphs=0` workaround, and
|
||||
`--split-mode graph` MoE note (PR 1080)
|
||||
`--split-mode graph` MoE note (PR 1080)
|
||||
|
||||
@@ -9,37 +9,33 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|
||||
|
||||
| Page | Owns |
|
||||
|---|---|
|
||||
| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
|
||||
| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
|
||||
| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
|
||||
| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
|
||||
| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
|
||||
| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |
|
||||
|
||||
## Retired endpoints
|
||||
|
||||
| Page | Owns |
|
||||
|---|---|
|
||||
| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
|
||||
| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
|
||||
| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
|
||||
| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
|
||||
| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
|
||||
| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
|
||||
| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
|
||||
| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
|
||||
| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
|
||||
| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
|
||||
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
|
||||
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
|
||||
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
|
||||
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
|
||||
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
|
||||
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
|
||||
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
|
||||
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
|
||||
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
|
||||
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |
|
||||
|
||||
## Pipeline Parallelism (series)
|
||||
|
||||
| Page | Owns |
|
||||
|---|---|
|
||||
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
|
||||
| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
|
||||
| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
|
||||
| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
|
||||
| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
|
||||
| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
|
||||
| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
|
||||
| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
|
||||
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
|
||||
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
|
||||
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
|
||||
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |
|
||||
|
||||
## Techniques, hardware, trials
|
||||
|
||||
@@ -47,43 +43,48 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|
||||
|---|---|
|
||||
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
|
||||
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
|
||||
| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
|
||||
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
|
||||
| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
|
||||
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
|
||||
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
|
||||
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
|
||||
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
|
||||
| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
|
||||
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
|
||||
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
|
||||
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
|
||||
| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
|
||||
| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
|
||||
| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
|
||||
| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
|
||||
| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
|
||||
| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
|
||||
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
|
||||
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
|
||||
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
|
||||
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
|
||||
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
|
||||
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
|
||||
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
|
||||
| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
|
||||
| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
|
||||
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
|
||||
| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
|
||||
| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
|
||||
| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
|
||||
| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
|
||||
| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
|
||||
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
|
||||
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
|
||||
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
|
||||
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
|
||||
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
|
||||
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
|
||||
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
|
||||
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
|
||||
| [[Quantization Production]] | — |
|
||||
| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
|
||||
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
|
||||
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
|
||||
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
|
||||
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
|
||||
| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
|
||||
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
|
||||
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
|
||||
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
|
||||
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
|
||||
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
|
||||
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
|
||||
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
|
||||
| [[swift15-campaign-worklog]] | — |
|
||||
| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
|
||||
| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
|
||||
| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
|
||||
| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
|
||||
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
|
||||
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
|
||||
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
|
||||
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
|
||||
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
|
||||
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
|
||||
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
|
||||
| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
|
||||
| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
|
||||
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
|
||||
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
|
||||
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | Owns |
|
||||
|---|---|
|
||||
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
---
|
||||
type: reference
|
||||
description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw."
|
||||
description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it."
|
||||
tags:
|
||||
- llm
|
||||
- rocm
|
||||
- frank
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-15
|
||||
---
|
||||
|
||||
@@ -239,3 +240,4 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
|
||||
`gl=45`. See the `set_rows` build bug section above.
|
||||
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
|
||||
retired the same day for the AGENT model). See [[frank-dragoon]].
|
||||
---
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
type: reference
|
||||
description: "`bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`."
|
||||
description: "Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict."
|
||||
tags:
|
||||
- llm
|
||||
timestamp: 2026-09-03
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Ling-3.0-flash Q6_K Trial on Dankclaw
|
||||
@@ -109,6 +109,13 @@ re-enable `frank-dragoon-ling` from the repo unit
|
||||
- The heap is effectively full (98.7 of 111.2 GiB). Do not raise the
|
||||
context much without checking `--list-devices` free MiB.
|
||||
|
||||
## The other path: vLLM-ROCm (never shipped)
|
||||
|
||||
A deployment plan existed for `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native`
|
||||
(INT4 W4A16, ~62 GB) served by vLLM-ROCm on Dankclaw. It never shipped; the
|
||||
llama.cpp Q6_K fork trial above is what actually ran. The decision table the
|
||||
plan carried is in git history.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[frank-dragoon]]
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
type: guide
|
||||
description: "This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on"
|
||||
tags:
|
||||
- llm
|
||||
- vllm
|
||||
- rocm
|
||||
timestamp: 2026-09-03
|
||||
---
|
||||
|
||||
# Ling-3.0-flash on Dankclaw — vLLM-ROCm Deployment Plan (never shipped)
|
||||
|
||||
This page was the plan for a vLLM-ROCm deployment of
|
||||
`jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on
|
||||
Dankclaw. The deployment never shipped. The actual Ling work used the
|
||||
llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]].
|
||||
|
||||
The trial service (`frank-dragoon-ling`) was retired and disabled under the boot
|
||||
policy on 2026-08-17, in the same redo that renamed `frank-dragoon` to the
|
||||
Qwen3.8-27B PP head ([[frank-redo-2026-08-17]]). Two fleet generations later the
|
||||
name belongs to the halogen flash-server endpoint ([[frank-dragoon]]). See git
|
||||
history for the full decision table.
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This page records the speculative-decoding benchmark history and the"
|
||||
description: "The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out."
|
||||
tags:
|
||||
- llm
|
||||
- speculative
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This page answers one question: **how do we make Qwen3.8-27B serve faster on"
|
||||
description: "How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought."
|
||||
tags:
|
||||
- llm
|
||||
timestamp: 2026-09-03
|
||||
|
||||
@@ -1,3 +1,13 @@
|
||||
---
|
||||
type: "reference"
|
||||
title: "Swift1.5 Campaign Worklog"
|
||||
description: "Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm."
|
||||
tags:
|
||||
- llm
|
||||
- campaign
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Swift1.5 w4dn8 Campaign — Work Log
|
||||
|
||||
Campaign branch `swift15-radiance-w4dn8`. Goal: convert
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: benchmark plan
|
||||
type: reference
|
||||
title: frank-dragoon ubatch sweep
|
||||
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
|
||||
description: "The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness."
|
||||
tags:
|
||||
- llama.cpp
|
||||
- rocm
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: vLLM Boot Forensics and Config Interrogation
|
||||
description: Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone), classifying the failure, and interrogating a live endpoint's true config and client contracts.
|
||||
description: "Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config."
|
||||
tags: [vllm, forensics, journalctl, podman, config, contracts]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user