Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Frank
|
d5934a6b47 | ||
 dank
|
40ad1f4a60 | ||
|
Frank
|
9de58ed6fd |
No files matched your search
@@ -477,333 +477,481 @@ linked runbooks.
|
|||||||
|
|
||||||
**PCIe / multi-GPU**
|
**PCIe / multi-GPU**
|
||||||
|
|
||||||
- **RCCL collectives need PCIe atomics on every participating GPU, and the failure never names
|
- **RCCL collectives need PCIe atomics on every participating GPU.** RCCL's
|
||||||
atomics.** The kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch them on
|
kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch
|
||||||
a device without atomics and the error surfaces as `hipErrorIllegalState` from `hipLaunchKernel`,
|
them when the device has no atomics, and the failure surfaces as
|
||||||
not as anything that names atomics. The value is consumed at driver PROBE time, so writing the
|
`hipErrorIllegalState` ("the operation cannot be performed in the present
|
||||||
AtomicOp Requester bit with `setpci` at runtime changes nothing. Check the CHAIN: `lspci -vv -s
|
state") from `hipLaunchKernel`, not as anything that names atomics. The
|
||||||
<gpu>` and every upstream bridge must show `AtomicOpsCap: Routing+` (a `Routing-` bridge is a
|
value is consumed at driver probe time, so writing the AtomicOp Requester
|
||||||
hardware wall), and the endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
|
bit with `setpci` at runtime changes nothing. Check the *chain*:
|
||||||
6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
|
`lspci -vv -s <gpu>` and every upstream bridge must show
|
||||||
chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
|
`AtomicOpsCap: Routing+` (a `Routing-` bridge is a hardware wall), and the
|
||||||
nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
|
endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
|
||||||
- **That wall was physical, and a board swap removed it on 2026-09-20.** dankserver took a board
|
6 set. AMD desktop platforms put one card on CPU root ports (atomics work)
|
||||||
whose CPU lanes bifurcate to x8/x8 Gen5: both cards on root ports, every bridge `Routing+`, BOTH
|
and the rest behind the chipset (usually not) — that asymmetry is what
|
||||||
endpoints `ReqEn+`. Bandwidth was never the constraint (a 27B TP=2 allreduce at batch 4 is ≈10 MB
|
blocked frank-medium's TP=2. A one-card collective proves nothing: NCCL
|
||||||
per token-step, under 0.4 ms at Gen5 x8); the collective choice is — TheRock's RCCL builds lack
|
short-circuits at `world_size=1`.
|
||||||
gfx1201 device code (`invalid device function`), which is why vLLM runs its R4D one-shot allreduce
|
- **That wall was physical, and it was fixed on 2026-09-20 by a board swap.**
|
||||||
here at all. Evidence matrix and the three-arm verdict: [[Dual-R9700-Cooperation]].
|
Both paths that put two cards on CPU root ports were: an M.2→PCIe x4 riser
|
||||||
- **When a GPU collective dies, read the runtime, not the framework.** The message that identified
|
into a *CPU-attached* M.2 slot, or a board whose CPU lanes bifurcate to
|
||||||
the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
|
x8/x8. dankserver took the second: **ASUS ProArt B850-Creator WiFi NEO**,
|
||||||
`AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
|
`PCIEX16_1` + `PCIEX16_2` at x8/x8 Gen5 on root ports `00:01.1` / `00:01.3`,
|
||||||
failing HIP call with its return code. [[PCIe Link and Atomics]]
|
every bridge now `Routing+` and **both endpoints `ReqEn+`** — the exact
|
||||||
- **Layer split is the no-collective path, on one vendor's cards and across two — and
|
precondition the old test plan named. Bandwidth was never the constraint (a
|
||||||
`--override-tensor` is what cannot ride with it.** Layers go in order and only the boundary
|
27B TP=2 allreduce at batch 4 is ≈10 MB per token-step, ≈1.4 ms at Gen4 x4
|
||||||
activation crosses PCIe once per forward pass, so it works wherever a collective cannot be
|
and under 0.4 ms at Gen5 x8).
|
||||||
dispatched (on an RCCL-less fork it beats tensor mode, which falls back to a butterfly allreduce),
|
**Proven on the same day.** vLLM's R4D one-shot allreduce runs TP=2 here and
|
||||||
and it is the cross-vendor answer too (a CUDA + Vulkan split held a real two-device graph). But
|
frank-medium serves on it; llama.cpp's RCCL path was built too
|
||||||
`--override-tensor <group>=<device>` sets `has_tensor_overrides()`, a REQUIRED term of the
|
(`GGML_HIP_RCCL=ON`) and its tensor mode measured — real collective, slower
|
||||||
scheduler's `pipeline_parallel` flag, and that flag sets `n_copies`: with `n_copies == 1` a
|
shape. Note the collective choice matters: TheRock's RCCL builds lack gfx1201
|
||||||
pre-allocated weight on a non-primary device has no backend and the graph aborts (reported with
|
device code (`invalid device function`), which is why R4D is the path vLLM
|
||||||
the leaf's op name `(NONE)`). `=CPU` is the only override target that works — the CPU backend is
|
uses at all. Evidence matrix and the measured three-arm verdict:
|
||||||
last in the list — and pure Vulkan changes nothing: the gate is the override, not the driver mix.
|
[[Dual-R9700-Cooperation]].
|
||||||
Pin the spec-decode drafter to one device; on a split its home is the card holding the target's
|
- **When a GPU collective dies, read the runtime, not the framework.** The
|
||||||
last layers. [[Llama Cpp Multi-Device Splits]] [[x4-pair-experiment]]
|
message that identified the cause came from `AMD_LOG_LEVEL=3`
|
||||||
- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design constraint, not a detail.**
|
(`Pcie atomics not enabled, hostcall not supported` → `AQL dispatch
|
||||||
Pinned H2D 6.72 / D2H 6.60 GB/s (≈85 % of the Gen4 x4 ceiling), pageable within 1 % of pinned —
|
failed!`), not from vLLM or RCCL. `AMD_LOG_LEVEL` costs one env var and
|
||||||
the copies are LINK-limited, so the usual "pageable costs 2×" rule does not apply; the x4 lives in
|
prints the failing HIP call with its return code.
|
||||||
the adapter or cable (card and port are both wider). For any engine that STREAMS weights across
|
- **The software path that puts both cards on ONE model without a collective:
|
||||||
it: a 3 MiB slab costs 474 µs, so a 111 GiB A3B model needing ~1.70 GiB of active experts per
|
llama.cpp's layer split.** On the old board vLLM could not do it in any of its
|
||||||
token lands at 3.7 tok/s against 20.46 tok/s with the experts COMPUTED in host RAM
|
parallelisms — TP needs an allreduce, PP needs the same p2p primitive, and both
|
||||||
(`-ot exps=CPU`, where the link carries activations only). Streaming loses; RAM does not care.
|
died on the card that could not dispatch an RCCL kernel. `--split-mode
|
||||||
[[x4-pair-experiment]]
|
layer` in llama.cpp needs no
|
||||||
|
collective: layers are distributed in order and only the boundary activation
|
||||||
|
crosses PCIe, once per forward pass. Measured here: 1991 t/s prefill /
|
||||||
|
18.8 tok/s decode layer-split, against 272 / 26.3 tensor (butterfly
|
||||||
|
fallback).
|
||||||
|
This was what `frank-medium` ran until 2026-09-20, when the board swap made
|
||||||
|
collectives legal and radiance TP=2 beat it on every axis (95.8 vs 45.7 t/s
|
||||||
|
decode, 0.21 vs 0.72 s TTFT, 259 vs 72 t/s burst-4) — see
|
||||||
|
[[frank-medium-radiance-mxfp4]]; the llama shape is the rollback target
|
||||||
|
[[frank-medium-llamacpp-dflash2]].
|
||||||
|
- **Layer split is also the cross-vendor answer — and `-ot` is what cannot be
|
||||||
|
combined with it.** On Dankclaw (2026-09-16) a CUDA 3090 + Vulkan iGPU layer
|
||||||
|
split ran a real two-device graph: `-dev CUDA0/Vulkan1 -sm layer -ts 0.15/0.85`
|
||||||
|
held 14,994 MiB on the 3090 and gave 27.2 tok/s with 268.5 t/s prefill at 64k
|
||||||
|
depth (iGPU alone: 25.8 / 231.8). What does NOT work is per-tensor placement
|
||||||
|
across devices: `--override-tensor <group>=<device>` sets
|
||||||
|
`has_tensor_overrides()`, a REQUIRED term of the scheduler's
|
||||||
|
`pipeline_parallel` flag (llama-context.cpp:433 — the only place in the tree
|
||||||
|
that reads it), and that flag is what sets `n_copies` in
|
||||||
|
`ggml_backend_sched_new`; with `n_copies == 1` a pre-allocated weight on a
|
||||||
|
non-primary device has no backend and the graph aborts (`ggml-backend.cpp:941`,
|
||||||
|
reported with the leaf's own op name `(NONE)`). `=CPU` is the only override
|
||||||
|
target that works, because the CPU backend is last in the list. Pure Vulkan
|
||||||
|
changes nothing here — the gate is the override, not the driver mix. See
|
||||||
|
[[x4-pair-experiment]].
|
||||||
|
- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design
|
||||||
|
constraint, not a detail.** `pcie-link-test.cu` on Dankclaw (2026-09-16,
|
||||||
|
services stopped): pinned H2D 6.72 GB/s, D2H 6.60 GB/s — 85 % of the Gen4 x4
|
||||||
|
ceiling — with async no better than sync and pageable within 1 % of pinned,
|
||||||
|
i.e. the copies are link-limited rather than staging-limited (the usual
|
||||||
|
"pageable costs 2x" rule does not apply here). The card is x16-capable and the
|
||||||
|
Strix Halo port beneath it is x8-capable (`LnkCap: 16GT/s, Width x8` on
|
||||||
|
`00:03.1`), so the x4 lives in the adapter or cable; the link idles at
|
||||||
|
2.5 GT/s x4 and trains up under load. Consequence for any engine that
|
||||||
|
*streams* weights across it: a 3 MiB slab costs 474 us, and a 111 GiB A3B
|
||||||
|
model needs ~1.70 GiB of active experts per token ⇒ **3.7 tok/s** (4.6 with a
|
||||||
|
20 % resident cache), against **20.46 tok/s** for the same model with the
|
||||||
|
experts *computed* in host RAM by llama.cpp (`-ot exps=CPU`, whence the link
|
||||||
|
carries activations only) and 25–26.5 on the iGPU. Streaming over PCIe loses;
|
||||||
|
computing in RAM does not care. FreeToken — expert LRU in host RAM, exactly
|
||||||
|
this streaming shape — also loads only modelopt block-FP8 safetensors for
|
||||||
|
`qwen3_5_moe`, so a Q4_K_M GGUF is unloadable there before bandwidth is even
|
||||||
|
the question. See [[x4-pair-experiment]].
|
||||||
|
|
||||||
|
|
||||||
**ROCm / GPU**
|
**ROCm / GPU**
|
||||||
|
|
||||||
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a trap.** With a missing
|
- **Layer split needs no collective** — layers go in order and only the boundary
|
||||||
`DST`, cp creates `DST` as a copy of `SRC` and drops one directory level. rocBLAS and hipBLASLt
|
activation crosses PCIe, so it is the fallback wherever a collective cannot be
|
||||||
look for their device code at a FIXED depth (`../lib/rocblas/library`,
|
dispatched (and on an RCCL-less fork it beats tensor mode, which falls back to
|
||||||
`../lib/hipblaslt/library/<arch>`), so the flattened copy fails at REQUEST time, not load time:
|
a butterfly allreduce). Pin the spec-decode drafter to one device: an unpinned
|
||||||
the engine reads the weights, answers `/health` and dies on the first completion with
|
draft under tensor mode aborts the KV build, and on a split its home is the
|
||||||
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch : gfx1201`. Declare
|
card holding the target's last layers. Deployed history:
|
||||||
the destination, then copy the contents.
|
[[frank-medium-llamacpp-dflash2]].
|
||||||
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
|
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a
|
||||||
anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
|
trap.** With a missing `DST`, cp creates `DST` as a copy of `SRC` and drops
|
||||||
and pinning the PLE tensor with `-ot` cost 23 → 13.4 t/s. [[Llama Cpp Serving Sizing]]
|
one directory level. rocBLAS and hipBLASLt look for their device code at a
|
||||||
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
|
fixed depth (`../lib/rocblas/library`, `../lib/hipblaslt/library/<arch>`),
|
||||||
with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
|
and the flattened copy fails at *request* time, not load time: the engine
|
||||||
`1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
|
reads the weights, answers `/health` and dies on the first completion with
|
||||||
tunable these; check with
|
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch
|
||||||
|
: gfx1201`. Declare the destination, then copy the contents.
|
||||||
|
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the
|
||||||
|
~50 GiB PLE table into anonymous memory → silent cgroup kill, dmesg is the
|
||||||
|
only evidence. mmap pages ARE page cache. Never pin the PLE tensor with
|
||||||
|
`-ot` (23 → 13.4 t/s).
|
||||||
|
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve
|
||||||
|
scripts export env files with `set -a`, so a tunable named `TEMP` leaks
|
||||||
|
into HIP (`TEMP=1.0` sent the JIT cache to path `1.0` and segfaulted every
|
||||||
|
load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a tunable
|
||||||
|
these; check with
|
||||||
`strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
|
`strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
|
||||||
- **`GGML_CUDA_FA_ALL_QUANTS` is dead; do not build for it.** Verified on this host 2026-10-19:
|
- **`-DGGML_CUDA_FA_ALL_QUANTS=ON` is required** for q8_0 KV + flash
|
||||||
master's `ggml/cmake/common.cmake` warns it is deprecated in favour of `GGML_CUDA_FA_QUANTS`, the
|
attention on CUDA builds; without it prefill collapses ~12x (95 vs 1300+
|
||||||
successor's DEFAULT pair list (`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16`) already compiles the
|
t/s, measured 2026-08-17).
|
||||||
quantized-KV kernels with the old flag OFF, and neither llama tree on this host references the old
|
- **Flash-Next MTP heads do not load on any current loader** — they need full
|
||||||
macro anywhere under `ggml/src/ggml-cuda/`. The old "~12× prefill collapse" belongs to the older
|
per-layer `blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified
|
||||||
source line. Prove a build's KV ability from its `CMakeCache.txt`. [[frank-zealot]]
|
3x). Don't re-attempt without new loader support.
|
||||||
- **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
|
- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on
|
||||||
`blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
|
qwen4exp needs `scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on
|
||||||
loader support.
|
the agentionai FAST layer set; the kingjones STRIX set works unpatched.
|
||||||
- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on qwen4exp needs
|
- **GPU budget on Dankclaw is the GTT aperture (~114 GiB), not the
|
||||||
`scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on the agentionai FAST layer set; the
|
rocm-smi "dedicated" figure.** The Radeon 8060S (gfx1151) reports
|
||||||
kingjones STRIX set works unpatched. [[Llama Cpp Serving Sizing]]
|
116,732 MiB (`llama-server --list-devices`;
|
||||||
- **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
|
`gtt_total=122402963456` in `/sys/class/drm/card1/device/`). The older
|
||||||
Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
|
"64 GiB, SVM-backed" lesson was already stale — the CIRU Orca predecessor
|
||||||
`/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
|
held 92.6 GiB of weights on the iGPU. `free`/`dmidecode` show the
|
||||||
`free`/`dmidecode` show the CPU-side carve-out — intentional, do not change. [[Dankclaw System Specs]]
|
CPU-side carve-out — intentional, do not change.
|
||||||
|
|
||||||
**Load / crash diagnosis**
|
**Load / crash diagnosis**
|
||||||
|
|
||||||
- Model load is slow (minutes cold) and health returns 503 while it runs: do NOT kill the process,
|
- Model load is slow (minutes cold); health returns 503 while loading; do
|
||||||
`deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On halogen the FIRST request after a (re)start
|
not kill the process. `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On
|
||||||
can go silent for ~45–60 s (`engine_unresponsive`, 503s) and then self-recovers — do not restart
|
halogen, the FIRST request after a (re)start can go silent ~45–60 s
|
||||||
over it. [[frank-dragoon]]
|
(health `engine_unresponsive`, 503s) and then self-recovers — do not
|
||||||
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` AND the kernel line via `journalctl -k` (a
|
restart over it.
|
||||||
segfault at address 8 in `libamdhip64` is a NULL unref in HIP); (2) run the serve script, then the
|
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` + kernel line via
|
||||||
raw binary — a different outcome isolates the env file; (3) `gdb -batch -ex run -ex "bt 25"
|
`journalctl -k` (segfault at address 8 in `libamdhip64` = NULL unref in
|
||||||
--args bash scripts/serve-<svc>.sh`, because llama-server buffers stdout and loses the log tail on
|
HIP); (2) run serve script, then raw binary — different outcome isolates
|
||||||
SIGSEGV: trust gdb, not the log; (4) `amdgpu: SVM mapping failed` spam is a symptom, not a root
|
the env file; (3) `gdb -batch -ex run -ex "bt 25" --args bash
|
||||||
cause. [[LLM Performance Diagnosis]]
|
scripts/serve-<svc>.sh` — llama-server buffers stdout and loses the log
|
||||||
- Journal signatures: a client abort logs `W srv stop: cancel task, id_task = N` while a natural stop
|
tail on SIGSEGV; trust gdb, not the log; (4) `amdgpu: SVM mapping failed`
|
||||||
logs only `slot release` with `truncated = 0`. The server never preempts a running generation;
|
spam is a symptom, not a root cause.
|
||||||
excess tasks queue when slots are busy.
|
- Journal signatures: client abort logs `W srv stop: cancel task, id_task =
|
||||||
|
N`; natural stop logs only `slot release` with `truncated = 0`. The server
|
||||||
|
never preempts a running generation; excess tasks queue when slots are
|
||||||
|
busy.
|
||||||
|
|
||||||
**Host memory / swap**
|
**Host memory / swap**
|
||||||
|
|
||||||
- **Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` of the old one**, or
|
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
||||||
live pages must land in RAM that `MemAvailable` overstates. Dankclaw's 64 GiB `/swap.img`
|
of the old one, or live pages must land in RAM that `MemAvailable`
|
||||||
(fstab-driven) protects the co-tenants, not an endpoint with pinned weights — paging the KV pool
|
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
||||||
is worse than the OOM it prevents. Host swap state is invisible to `deploy.sh` (no repo file):
|
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
|
||||||
`swapon --show` before reasoning about headroom. [[Swap Configuration]]
|
the KV pool is worse than the OOM it prevents.
|
||||||
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults to half of RAM, so
|
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
||||||
a manual raise vanished with a power failure, frank-archon's pinned-ledger preflight refused the
|
`swapon --show` before reasoning about headroom.
|
||||||
start, and `Restart=always` re-ran that refusal 6,919 times in ~20 h — **a unit in
|
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults
|
||||||
`activating (auto-restart)` that exits 1 in 50 ms with no container and no engine log is a
|
to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the
|
||||||
PREFLIGHT refusal, not an engine fault.** Repo law now: `scripts/ttm-preflight.sh` raises the knob
|
2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB,
|
||||||
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only after a write the
|
frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start,
|
||||||
kernel rejected. Never prescribe a manual sysfs write as the recovery procedure for a service that
|
and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in
|
||||||
can boot before a human reaches it. [[frank-archon]]
|
`activating (auto-restart)` that exits 1 in 50 ms with no container and no
|
||||||
- **Silent RAM corruption outranks every software theory.** frank-archon's long-conversation
|
engine log is a PREFLIGHT refusal, not an engine fault. Repo law now:
|
||||||
repetition/garbage was marginal DDR5 at 4800 MT/s on four DIMMs, fixed by dropping to 4600.
|
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob
|
||||||
Consumer non-ECC raises NO MCE for a silent flip, and a short in-band pattern check (60 s over
|
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only
|
||||||
8 GiB of 246 GiB) is not a stability proof — both instruments said "ruled out" while the RAM was
|
after a write the kernel rejected. Never prescribe a manual sysfs write as the
|
||||||
the cause, and "4800 is the JEDEC floor" is a rating, not a margin guarantee. When the fault is
|
recovery procedure for a service that can boot before a human reaches it.
|
||||||
data-shaped (NaN states, garbage tokens) and a software revert does not move it, price in the RAM
|
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
||||||
before the runtime; a revert tested under the same marginal hardware is itself confounded.
|
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
||||||
[[frank-archon]]
|
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
||||||
|
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
||||||
|
a short in-band pattern check (60 s, 8 GiB of 246 GiB) is not a stability
|
||||||
|
proof — combined, those two instruments said "ruled out" while the RAM
|
||||||
|
was the cause. "4800 is the JEDEC floor on 5600-rated DIMMs" is a rating,
|
||||||
|
not a margin guarantee. When the fault is data-shaped (NaN states,
|
||||||
|
garbage tokens) and a software revert doesn't move it, price in the RAM
|
||||||
|
before the runtime — and remember a revert tested under the same marginal
|
||||||
|
hardware is itself confounded. See the runbook's 2026-10-05 section.
|
||||||
|
|
||||||
**Runtime-specific guards**
|
**Runtime-specific guards**
|
||||||
|
|
||||||
- **Rootless podman cannot RAISE rlimits, so `--ulimit nofile=1048576` dies at container CREATE**,
|
- **Rootless podman cannot RAISE rlimits, and `--ulimit nofile=1048576`
|
||||||
not at runtime: runc reports `error setting rlimit type 7: operation not permitted` (type 7 =
|
dies at container create, not at runtime.** runc reports
|
||||||
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's ceiling (524288 on
|
`error setting rlimit type 7: operation not permitted` (type 7 =
|
||||||
Dankclaw). Inherited-equal limits are fine, so `memlock=-1:-1` works iff the unit grants
|
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's
|
||||||
`LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
|
hard limit — on Dankclaw that ceiling is 524288. Inherited-equal limits
|
||||||
any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
|
are fine, so `memlock=-1:-1` works iff the unit grants
|
||||||
before adding flags. [[frank-dragoon]]
|
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
||||||
- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a file to a
|
podman to runc without any raise). Upstream halogen's own run lines pass
|
||||||
loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is ftype 115 over 311 tensors of type 103
|
NO `--ulimit` at all — copy vendor run lines before adding flags
|
||||||
(`GGML_TYPE_Q8_0_ROCMFPX`), so stock llama.cpp and gufo refuse it at load and the fork was not a
|
(measured 2026-10-05 on the frank-dragoon cutover).
|
||||||
preference but the only door. Counting tensor types out of the GGUF KV section takes one python
|
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
||||||
read and settles a runtime argument in minutes. Build identity matters as much as the fork: the
|
`third_party/composable_kernel/include/ck/config.h` generated manually
|
||||||
2026-08-25 ROCmFPX builds SIGSEGV in `ggml_compute_forward_set_rows` on this same file, so
|
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
||||||
`scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA. [[GGUF Compatibility]]
|
non-greedy requests and reasoning budgets. Re-test chat before production.
|
||||||
- **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
|
- **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
|
||||||
(the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
|
stock binaries fail the draft load with a tensor-shape mismatch.
|
||||||
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
|
- **A GGUF's custom tensor type picks the runtime — read the histogram before
|
||||||
measures a crash instead of the absence of drafting. [[frank-zealot]]
|
promising a file to a loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is
|
||||||
- **An orphaned scratch server taxes the whole host.** A bench child left alive by a client-side
|
ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`); stock
|
||||||
exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
|
llama.cpp and gufo refuse it at load, so ROCmFPX was not a preference but
|
||||||
every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
|
the only door. Counting tensor types out of the GGUF KV section takes one
|
||||||
back before believing a number — and kill by pid, never `pkill -f llama-server`.
|
python read and settles a runtime argument in minutes. The build identity
|
||||||
[[LLM Bench Windows and Measurement]]
|
matters as much as the fork: the 2026-08-25 ROCmFPX builds SIGSEGV in
|
||||||
- Retired-runtime traps that can come back: **Kairic Edge** needs
|
`ggml_compute_forward_set_rows` on this same file, so
|
||||||
`third_party/composable_kernel/include/ck/config.h` generated manually (untracked, so `git clean`
|
`scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA.
|
||||||
removes it) and its greedy-argmax fastpath rejects non-greedy requests and reasoning budgets, so
|
- **Speculative-decode flags move as a SET.** Handing the ROCmFPX fork
|
||||||
re-test chat before production; the **FastMTP sidecar** must be commit `4df29be4` + the HauhauCS
|
`--model-draft` with no `--spec-type` (the naive way to build a "no MTP"
|
||||||
patch, since stock binaries fail the draft load with a tensor-shape mismatch.
|
bench arm) segfaulted at load, core dumped, no diagnostic (2026-10-02). A
|
||||||
|
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND
|
||||||
|
`--spec-type`, or it measures a crash instead of the absence of drafting.
|
||||||
|
- **An orphaned scratch server taxes the whole host.** A bench child left
|
||||||
|
alive by a client-side exception kept ~26 GiB of weights resident on a
|
||||||
|
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
||||||
|
Record the child pid and kill it in a `finally`, then prove `free` came back
|
||||||
|
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
||||||
|
which also hits frank-zealot.
|
||||||
|
|
||||||
**llama.cpp serving**
|
**llama.cpp serving**
|
||||||
|
|
||||||
- **MTP is a property of the FILE, not of the flags.** The `-mtp` export carries the head as
|
- **MTP is a property of the FILE, not of the flags.** The `-mtp` GGUF build
|
||||||
`blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`); `--spec-type draft-mtp` sets
|
carries the head as `blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`);
|
||||||
`load_mtp` and the draft context runs on the MAIN model's weights (`shares_model`), so there
|
`--spec-type draft-mtp` then sets `load_mtp`, and the draft context runs on
|
||||||
is no second copy of a 12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
the MAIN model's weights (`shares_model`), so there is no second copy of a
|
||||||
tensors at all, and pointing `--model-draft` at that file loads it twice. [[frank-zealot]]
|
12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
||||||
- **What can be chained: ONE context-backed drafter plus any number of draftless ngram types.**
|
tensors at all, and pointing `--model-draft` at the file would load it twice.
|
||||||
`--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
|
- **What can be chained, what cannot, and what a chain costs on a hybrid.**
|
||||||
`ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
|
`--spec-type a,b` builds every enabled implementation and chains drafting
|
||||||
`draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
|
through them, in a fixed internal order (ngram types first, then the
|
||||||
legal (measured 2.6x the drafter alone on prompt-echo traffic) but `draft-mtp,draft-dflash`
|
drafters — CLI order is irrelevant). The gate is `ctx_dft`: **draftless**
|
||||||
dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
|
types (the ngram family) are added unconditionally, while `draft-mtp`,
|
||||||
chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
|
`draft-dflash`, `draft-dspark` and `draft-eagle3` all require a draft
|
||||||
hybrid measured −15 % prose / −7 % block-copy, because adding the ngram layer logs
|
CONTEXT. So a chain may hold one context-backed drafter plus any number of
|
||||||
`recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
|
ngram types — `draft-dflash,ngram-mod` is legal and measured (2.6x the
|
||||||
MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
|
drafter alone on prompt-echo traffic, byte-identical to it on prose at the
|
||||||
[[frank-dragoon]]
|
fleet's 48/64 window) — but `draft-mtp,draft-dflash` dies at load: *"context
|
||||||
- **A drafter's KV scales with the TARGET's context, and its window policy decides the price.**
|
type MTP requested but model doesn't contain MTP layers"*, because MTP
|
||||||
The draft context is built at `llama_n_ctx(ctx_tgt)` with no flag to cap it, so a drafter
|
builds its context against whatever model the draft context holds. Two
|
||||||
declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
|
context-backed types cannot share one process, and MTP's head only loads
|
||||||
first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
|
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
||||||
drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
|
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
|
||||||
[[Llama Cpp Multi-Device Splits]]
|
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
||||||
- **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
|
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
||||||
compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
|
disabling rollback support`, and MTP verification depends on that rollback,
|
||||||
wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
|
so acceptance drops on ALL traffic even where the matcher fires. Measure
|
||||||
cost against what the batch knobs leave for compute. Measure the arm, never the flag.
|
chains per architecture, never by analogy.
|
||||||
[[frank-zealot]]
|
- **A drafter's KV scales with the TARGET's context, and its window policy
|
||||||
- **A quant's scale overhead belongs in its byte rate; doubling a measured figure to "get the
|
decides the price.** The draft context is built at `llama_n_ctx(ctx_tgt)` with
|
||||||
f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element (32 codes + one 2-byte scale per
|
no flag to cap it, so a drafter declaring NO sliding window caches the whole
|
||||||
32-block), f16 is exactly 2.0 ⇒ the ratio is **1.882×**, not 2×. Derive byte rates from element
|
window on `--spec-draft-device` and can OOM the first draft graph capture
|
||||||
count × block layout, then check them against the boot ledger — this repo carried an invented
|
AFTER `/health` has answered. Read `sliding_window` out of the drafter's GGUF
|
||||||
2× number for weeks. Current KV math: [[frank-zealot]]
|
header before sizing, and prefer a skewed `-ts` over a context cut. Worked
|
||||||
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is BACKEND-SPECIFIC — never
|
example: [[frank-medium-llamacpp-dflash2]].
|
||||||
carry it across hosts.** Its compute buffers land about TWICE as heavy on the card that holds
|
- **A draft context bills its own f16 KV over the WHOLE POOL, and on a
|
||||||
the output layers and the drafter, so a split balanced at one batch is unbalanced at another:
|
single-card endpoint that squeeze can cost more than the drafter wins.** With
|
||||||
re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
|
`--parallel N` the drafter pays the pool, not the slot, plus its own compute
|
||||||
large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
|
buffers. So the MTP question is never "does the file carry the head" but "what
|
||||||
model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
|
do draft KV + buffers cost against what the batch knobs leave for compute" —
|
||||||
control that proves the knob, not the traffic, moved the number. The 2× R9700 split peaks at
|
a per-shape answer. Measure the arm, never the flag; the booted arm table is
|
||||||
512; the CUDA 3090 has no interior optimum up to 2048. When a bench IS ordered, A/B it by
|
on [[frank-zealot]].
|
||||||
restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
|
- **A quant's scale overhead belongs in its byte rate; doubling a measured
|
||||||
port: one card holds one server. Ladders: [[frank-medium-llamacpp-dflash2]] and
|
figure to "get the f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element
|
||||||
[[frank-zealot]].
|
(32 codes + one 2-byte scale per 32-block), f16 is exactly 2.0 ⇒ the ratio is
|
||||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
|
**1.882×**, not 2×. Derive byte rates from element count × block layout, then
|
||||||
boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
|
check them against the boot ledger — this repo carried an invented 2× number
|
||||||
by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
|
for weeks. Current KV math: [[frank-zealot]].
|
||||||
[[Llama Cpp Multi-Device Splits]]
|
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is
|
||||||
- **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
|
BACKEND-SPECIFIC — never carry it across hosts.** Its compute buffers land
|
||||||
matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
|
about TWICE as heavy on the card that holds the output layers and the drafter,
|
||||||
only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
|
so a split balanced at one batch is unbalanced at another: re-measure the
|
||||||
costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
|
balance after any batch change. The logical `--batch-size` is nearly free.
|
||||||
too short and it proposes on ordinary prose fragments, costing verification without acceptance.
|
A large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit
|
||||||
[[Llama Cpp Multi-Device Splits]]
|
— no totals model can see that. Ladders: [[frank-medium-llamacpp-dflash2]]
|
||||||
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
|
(R9700 split) and [[frank-zealot]] (CUDA 3090).
|
||||||
the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
|
- **On the CUDA 3090 that same knob has no interior optimum up to 2048.** A
|
||||||
prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
|
prompt shorter than the ubatch cannot see the knob at all — that is the
|
||||||
`contextWindow` MUST follow every reslice. Both slot directions cost or buy window, never
|
control that proves the knob, not the traffic, moved the number. When a bench
|
||||||
pool; shorter slots decode slightly faster. Measured reslices: [[frank-zealot]].
|
IS ordered, A/B it by restarting the SERVICE through a transient
|
||||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj costs ~1.5× its file
|
`FRANK_ZEALOT_ENV_FILE` drop-in, never a scratch port: one card holds one
|
||||||
size** (the encoder's activation and compute buffers ride with it). Subtracting two booted rows
|
server. Numbers: [[frank-zealot]].
|
||||||
isolates it exactly; extrapolating weights+KV+"a bit for buffers" is what leaves 1.1 GiB free
|
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
||||||
when you meant 2.7. Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
size** (a DeltaNet boundary and a full-attention boundary can differ by ~1 GiB
|
||||||
`PARALLEL × 4096`. **The projector is NOT billed in llama.cpp's own fit check**: the engine
|
each with its own KV), so balance by measurement. An affine fit reproduces the
|
||||||
prints its worst case and fits WITHOUT it, so price a shape from settled `nvidia-smi` after a
|
deployed point and overstates the gaps in between.
|
||||||
real boot — the projector, and any drafter context, lands after `/health` would already have
|
[[frank-medium-llamacpp-dflash2]].
|
||||||
answered (same class as the DSpark drafter OOM). Two-boot table: [[frank-zealot]].
|
- **ngram-mod prints no startup banner.** The load log shows the drafter's
|
||||||
- **`images` and `image_url` are NOT interchangeable: only the content part reaches the model.**
|
`adding speculative implementation 'draft-dflash'` and nothing at all for the
|
||||||
`--mmproj` loads, `usage.prompt_tokens` rises, and the model still invents an answer if the
|
matcher, so a chain can look unengaged while it is working. Identify it
|
||||||
client used the legacy top-level `images` field on `/v1/chat/completions` — frank-zealot
|
behaviourally (`draft_n` > 0 only where text repeats) or by the per-task
|
||||||
answered "Brown." for a solid-red PNG that way and "Red." through the OpenAI `image_url` part.
|
`draft acceptance = ... mean len = ...` line. It costs no VRAM (a 64 MiB
|
||||||
The failure is silent because a hallucinated color is a plausible answer; probe with a solid
|
host-RAM hash table) and its window decides which traffic it can see: too
|
||||||
color and expect an exact word. [[frank-zealot]]
|
short and it proposes on ordinary prose fragments, which costs verification
|
||||||
- **`--jinja` is what `tools` needs.** A subagent died with `500 tools param requires --jinja
|
without acceptance.
|
||||||
flag` on the ik_llama.cpp runtime that served frank-dragoon: every tool-using agent on that
|
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
|
||||||
endpoint was broken and the failure read like a model fault. Mainline llama.cpp defaults
|
per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
|
||||||
`--jinja` on and the serve scripts pin it; halogen also accepts `tools` and returns
|
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT with
|
||||||
`tool_calls`. Check tool support before routing a role at an endpoint.
|
a 400 even when the pool has room, so `models.yml` `contextWindow` must follow
|
||||||
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's template accepts
|
every reslice. Both slot directions cost or buy window, never pool; shorter
|
||||||
only `xhigh`, `medium` and `low` for `reasoning_effort` and raises on anything else — HTTP 500,
|
slots decode slightly faster (less attention to scan, one fewer recurrent
|
||||||
not a clamp — so the harness's levels must be folded onto that set in `models.yml`
|
state). Measured reslices: [[frank-zealot]].
|
||||||
(`compat.reasoningEffortMap`). A role whose level is unmapped fails on its FIRST request and
|
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
||||||
the failure names Jinja, not the model: every subagent on `frank-zealot` died with
|
costs ~1.5× its file size** (the encoder's activation and compute buffers ride
|
||||||
`Unexpected reasoning effort high` until `high: xhigh` was added. Halogen accepts the wider
|
with it). Subtracting two booted rows isolates it exactly; extrapolating
|
||||||
set, so the two entries legitimately differ.
|
weights+KV+"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7.
|
||||||
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe reads as
|
Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||||
blindness.** A probe at `max_tokens: 64` returned `content: ""` with `finish_reason: "length"`
|
`PARALLEL × 4096`. The two-boot table: [[frank-zealot]].
|
||||||
and half a thought in `reasoning_content` while the image HAD reached the model (322 prompt
|
- **`images` and `image_url` are NOT interchangeable: only the content part
|
||||||
tokens vs 42 for the text alone). Budget ≥512 for any thinking model you probe for vision.
|
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
||||||
- **The FIRST request after a boot is a warmup artifact**, and the 3× spread on one shape makes
|
model still invents an answer if the client used the legacy top-level
|
||||||
graph-capture cost read as a model fault. Benchmark the second request or later.
|
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
|
||||||
[[frank-zealot]]
|
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
||||||
- **A fork build hides its own dtype laws.** Vendor builds carry their own compile options, so
|
failure is silent because a hallucinated color is a plausible answer. Probe
|
||||||
prove a build's KV behaviour from the KV buffer line in the boot ledger and its
|
vision with a solid color and expect an exact word.
|
||||||
`CMakeCache.txt` — never from the build flags you remember passing. [[frank-zealot]]
|
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
||||||
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed**: without it
|
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
||||||
the engine answers **501 Not Implemented** on `/metrics`, so the scrape fails while `/health`
|
then served frank-dragoon: every tool-using agent on that endpoint was broken
|
||||||
answers 200 and `LlmEndpointDown` fires beside a healthy service. halogen serves `/metrics`
|
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
||||||
natively, so the asymmetry reads as "only the 3090 is broken". Diagnose with `lastError` from
|
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
||||||
`/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot
|
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
||||||
re-stamps `startsAt` on an alert that has fired for weeks. [[Observability]] [[frank-zealot]]
|
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
||||||
- **A leftover bench drop-in silently hijacks the next deploy, and no health gate sees it.** The
|
option and the same ~12x prefill collapse described in the ROCm note above.
|
||||||
sanctioned bench-window method (a transient `Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`
|
f16 KV needs nothing; frank-zealot ran f16 for that reason until
|
||||||
drop-in sourcing the repo env) outlives the bench unless retired: a leftover
|
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
||||||
`frank-zealot.service.d/ptq1.conf` made the cutover deploy boot a **different model at a
|
endpoint could take q8_0 KV.
|
||||||
different shape** while `/health` answered 200, `/v1/models` echoed the alias (a flag, not a
|
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
||||||
fact) and the Playwright gate passed 33/33. After deploying any recently-benched LLM service:
|
`-dev` and `-ts` follow this: `-dev CUDA0,Vulkan1` is two *single-device*
|
||||||
`systemctl show <svc> -p DropInPaths` (must be empty), then
|
configurations, never one two-device run, and `-ts 0.85,0.15` is two sweep
|
||||||
`tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger — and retire drop-ins
|
values each becoming `[0.85, 0, …]` / `[0.15, 0, …]`, i.e. device 0 takes
|
||||||
to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`. [[frank-zealot]]
|
everything. Write `-dev A/B -ts x/y`. This artifact produced a wrong fleet
|
||||||
- **A bench verdict that never reaches the repo is indistinguishable from a rogue arm.** That
|
conclusion on 2026-09-16 — "CUDA + Vulkan cannot serve one model", with two
|
||||||
same arm served production for 24 minutes with real numbers (~48 t/s decode, 801 t/s prefill at
|
bogus OOMs that were really "all 111 GiB on the 24 GiB card" — and the bench
|
||||||
141k) that lived only in `/tmp` env files and journald, so the next agent read the surviving
|
table's own `dev` column was printing the truth the whole time. Any
|
||||||
drop-in as an unfinished experiment and reported a hijack. Write the measurement into the
|
placement-dependent number needs an independent check: sample the other
|
||||||
endpoint page and the arm script's header in the session that takes it — `/tmp` and the journal
|
|
||||||
are not records.
|
|
||||||
- **`llama-bench` separates a LIST with `/` and SWEEP values with `,`** — for both `-dev` and
|
|
||||||
`-ts`. `-dev CUDA0,Vulkan1` is two *single-device* configurations, never one two-device run,
|
|
||||||
and `-ts 0.85,0.15` is two sweep values each putting everything on device 0. Write
|
|
||||||
`-dev A/B -ts x/y`. This artifact produced a wrong fleet conclusion ("CUDA + Vulkan cannot serve
|
|
||||||
one model") from two bogus OOMs, and the bench table's own `dev` column was printing the truth
|
|
||||||
the whole time. Any placement-dependent number needs an independent check: sample the other
|
|
||||||
device's memory (`nvidia-smi`) while the run is loaded.
|
device's memory (`nvidia-smi`) while the run is loaded.
|
||||||
|
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's
|
||||||
|
template accepts only `xhigh`, `medium` and `low` for `reasoning_effort` and
|
||||||
|
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
||||||
|
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
||||||
|
A role whose level is unmapped fails on its FIRST request, and the failure
|
||||||
|
names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||||
|
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
||||||
|
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
||||||
|
differ.
|
||||||
|
- **The vision projector is NOT billed in llama.cpp's own fit check.** The
|
||||||
|
engine prints `[mtmd] estimated worst-case memory usage of mmproj is
|
||||||
|
873.10 MiB` and then fits WITHOUT it: frank-zealot's Bonsai-2 boot projected
|
||||||
|
21,439 MiB and settled at 22,664 (2026-10-17). Price a 3090-class shape from
|
||||||
|
settled `nvidia-smi` after a real boot, never from
|
||||||
|
`model + context + compute` — the projector (and any drafter context) lands
|
||||||
|
after `/health` would already have answered. Same failure class as the
|
||||||
|
DSpark drafter OOM.
|
||||||
|
- **The FIRST request after a boot is a warmup artifact.** Measured on
|
||||||
|
Bonsai-2/3090: 2.6 t/s prefill and 37.7 t/s decode on request #1, then
|
||||||
|
1,258 t/s prefill and 54.7 t/s (per stream, two concurrent) on the next — a
|
||||||
|
3× spread on the same shape. Benchmark the second request or later, or the
|
||||||
|
graph-capture cost reads as a model fault.
|
||||||
|
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe
|
||||||
|
reads as blindness.** The Bonsai-2/`image_url` probe at `max_tokens: 64`
|
||||||
|
returned `content: ""` with `finish_reason: "length"` and half a thought in
|
||||||
|
`reasoning_content` — the image HAD reached the model (322 prompt tokens vs
|
||||||
|
42 for the text alone). At 512 tokens the same image answers "Red". Budget
|
||||||
|
≥512 for any thinking model you probe for vision.
|
||||||
|
- **Quantized KV is not the only dtype law a fork build can hide.** Vendor
|
||||||
|
builds carry their own compile options: frank-zealot's PrismML prebuilt
|
||||||
|
served `--cache-type-v q8_0` under flash attention at the exact 34,816
|
||||||
|
B/token (the `GGML_CUDA_FA_ALL_QUANTS` door), so a build decision must be
|
||||||
|
proven by the KV buffer line in the boot ledger, not by the build flags you
|
||||||
|
remember passing.
|
||||||
|
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed.**
|
||||||
|
Without it the engine answers **501 Not Implemented** on `/metrics`, so the scrape
|
||||||
|
fails while `/health` answers 200 and `LlmEndpointDown` fires beside a healthy
|
||||||
|
service — that is how `frank-zealot` sat red across four payload generations
|
||||||
|
(the oldest retained sample of `up{job="frank-zealot"}`, 2026-10-11, is already
|
||||||
|
0). halogen serves `/metrics` natively, so the asymmetry reads as "only the 3090
|
||||||
|
is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list,
|
||||||
|
and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an
|
||||||
|
alert that has been firing for weeks.
|
||||||
|
- **A leftover bench drop-in silently hijacks the next deploy, and no health
|
||||||
|
gate sees it.** The sanctioned bench-window method — a transient
|
||||||
|
`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env` drop-in, the arm env
|
||||||
|
`source`ing the repo env and then overriding it — outlives the bench unless
|
||||||
|
it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` —
|
||||||
|
the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover
|
||||||
|
deploy boot a **different model at a different shape** (3 slots of 133,888,
|
||||||
|
not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias
|
||||||
|
(a flag, not a fact) and the Playwright gate passed 33/33. After deploying any
|
||||||
|
recently-benched LLM service: `systemctl show <svc> -p DropInPaths` (must be
|
||||||
|
empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot
|
||||||
|
ledger — and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` +
|
||||||
|
`daemon-reload`.
|
||||||
|
- **A bench verdict that never reaches the repo is indistinguishable from a
|
||||||
|
rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real
|
||||||
|
numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only
|
||||||
|
in `/tmp` env files and journald, so the next agent on the host read the
|
||||||
|
surviving drop-in as an unfinished experiment and reported a hijack. Write the
|
||||||
|
measurement into the runbook and the arm script's header in the session that
|
||||||
|
takes it — `/tmp` and the journal are not records.
|
||||||
|
|
||||||
**Ops**
|
**Ops**
|
||||||
|
|
||||||
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
|
- Never `pkill -f` a pattern that appears in your own command line — it kills
|
||||||
Use `pkill -x <exact-name>`.
|
your ssh session. Use `pkill -x <exact-name>`.
|
||||||
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
|
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in
|
||||||
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
|
committed scripts (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts
|
||||||
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
|
without it.
|
||||||
`GroupAdd=keep-groups`, plus the udev rule that makes those nodes world-writable — user-unit
|
- ComfyUI on Dankclaw runs with **GPU access** (`/dev/kfd` + `/dev/dri` via
|
||||||
spawns carry no supplementary groups) and must NOT pass `--cpu`: the gfx1151 ROCm wheels exist
|
`AddDevice`, `GroupAdd=keep-groups`, plus the udev rule that makes those nodes
|
||||||
and a stale device-less container was found restarted 9,146 times. [[ComfyUI]]
|
world-writable — user-unit spawns carry no supplementary groups). It must NOT
|
||||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun
|
pass `--cpu`: the gfx1151 ROCm wheels exist and the unit's `--cpu` era ended
|
||||||
service. [[SearXNG]] [[Gluetun]]
|
2026-09-16, when a stale device-less container was found restarted 9,146
|
||||||
- **Dankclaw's kernel command line is a repo file.** It boots **`iommu=pt`** since 2026-10-18,
|
times (`RuntimeError: No HIP GPUs are available`).
|
||||||
tracked in `dankclaw/host/grub`, and moves ONLY through `scripts/set-dankclaw-bootline.sh`
|
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
||||||
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by hand-editing
|
ports off the gluetun service.
|
||||||
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
|
- Dankclaw boots **`iommu=pt`** since 2026-10-18. The boot line is tracked in
|
||||||
says identity mode costs nothing, so the XDNA2 NPU returned for free.
|
`dankclaw/host/grub` and moves ONLY through `scripts/set-dankclaw-bootline.sh`
|
||||||
[[dankclaw-iommu-translation-off]]
|
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by
|
||||||
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
|
hand-editing `/etc/default/grub`. It replaced `amd_iommu=off` (2026-09-11),
|
||||||
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
|
adopted on the vendor's 13–16 % prefill figure and never measured here: the
|
||||||
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
|
local A/B on byte-identical traffic says identity mode costs **nothing**
|
||||||
[[Quadlet Lifecycle]]
|
(+2.0 %, 1,616 t/s cold prefill), so the XDNA2 NPU returned for free —
|
||||||
- **A dead rootless podman pause process breaks ALL container management on the host, and
|
`amdxdna` binds `c6:00.1` (PASID width 10, identity group 26) and the old
|
||||||
`podman system migrate` panics in 5.7.0.** Every `podman` command fails with
|
`err=-19` only ever asked for an enabled IOMMU. **AMD PerfOpt** (iommu tree
|
||||||
`invalid internal status`, every start fails binding the bridges
|
2026-09-24, Linux 7.4) still skips the page walk outright for the privileged
|
||||||
(`failed to bind udp listener on 10.89.x.1:53`), every `Restart=always` quadlet loops — while
|
iGPU, but it arms only in the identity domain and 26.04 caps at `7.0.0-38`.
|
||||||
container storage stays intact. What is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
|
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
|
||||||
token) and the rootless netns; deleting the stale token is not enough. Repair: **reboot**
|
only the CURRENT instance; history across recreations lives in
|
||||||
(`/run/user` clears and the quadlets return on their own — check `Linger=yes` and
|
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
|
||||||
`WantedBy=default.target` first). The 2026-09-16 occurrence had no OOM in `journalctl -k`:
|
(per-user storage — `sudo podman logs` finds nothing).
|
||||||
treat the pause process as host state that can die for its own reasons. [[Quadlet Lifecycle]]
|
- **A dead rootless podman pause process breaks ALL container management on the
|
||||||
- **A container cannot reach a `Network=host` service at the host's LAN IP**: netavark DNATs
|
host, and `podman system migrate` panics in 5.7.0.** Symptoms: every `podman`
|
||||||
*published* ports only, so the host bind and the `10.89.1.1` gateway are both refused from a
|
command fails with `invalid internal status … could not find any running
|
||||||
bridge container while the published port answers. Use `host.containers.internal`
|
process`, every container start fails binding podman's bridge addresses
|
||||||
(169.254.1.2). This 502'd `hermes.franksdank.cloud` until its Caddyfile upstream changed.
|
(`Error starting server failed to bind udp listener on 10.89.x.1:53: Cannot
|
||||||
[[Container Host Reachability]]
|
assign requested address`), and every `Restart=always` quadlet loops. Container
|
||||||
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as `.partNofM`
|
storage is intact; what is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
|
||||||
fragments** while the smaller rungs are single files, so filtering `endswith(".gguf")` makes a
|
token, not a pid) and the rootless netns holding the bridges — deleting the
|
||||||
whole quant class look absent. Fetch the parts, verify each against the Hub's LFS sha256,
|
stale token is not enough, and `podman system migrate` segfaults in
|
||||||
concatenate in numeric order. [[Model Swap and Rollback]]
|
`teardownNetworkBackend`. Repair: **reboot** (`/run/user` clears and all 28
|
||||||
- **A fetch script must `-f` and verify, or it reports success on an access error.**
|
quadlets return on their own — check `Linger=yes` and `WantedBy=default.target`
|
||||||
`curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
|
first). Seen 2026-09-16, hours into heavy GPU work on the same host — but the
|
||||||
repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
|
trigger is NOT established (`journalctl -k` recorded no OOM kill; the kernel
|
||||||
seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
|
had no other word on it). Treat the pause process as host state that can die
|
||||||
declared size, and check the LFS sha256 (`scripts/download-frank-medium-gguf.sh` does all three;
|
for its own reasons, and reach for the reboot when it does.
|
||||||
ad-hoc fetches tend not to). Gated repos 403 even with the host's HF token until that account
|
- **A container cannot reach a `Network=host` service at the host's LAN IP.**
|
||||||
accepts the gate. [[Model Swap and Rollback]]
|
netavark DNATs *published* ports only: from a bridge container
|
||||||
- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup campaign six
|
`192.168.0.143:<published>` answers while `192.168.0.143:<host bind>` and the
|
||||||
read-only scouts worked the same pages: one classified era pages as stale *because* they
|
gateway `10.89.1.1` are both refused. Use `host.containers.internal`
|
||||||
document retired payloads (that is an era page's content), one reported files that do not exist,
|
(169.254.1.2). This is what 502'd `hermes.franksdank.cloud` until the Caddyfile
|
||||||
one reported duplications that grep disproved (`457.9 t/s` and the ping-RTT table are solo-held
|
upstream was changed on 2026-09-16 (its vhost comment had claimed the LAN IP
|
||||||
by `Ethernet Bandwidth Math`), one cited a line number past the end of the file it judged, and
|
was open from that bridge — it never was).
|
||||||
one drafted an Eras table with invented windows, payloads and rollback paths. Re-verify every
|
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as
|
||||||
claimed duplicate, every claimed staleness and every quoted holder against the file and today's
|
`.partNofM` fragments.** `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF`
|
||||||
fleet state before editing; a scout yield buys no evidence.
|
carries Q4_K_M (37.00 + 37.00 + 36.97 = 110.97 GiB) and Q4_K_S (35.00 + 35.00
|
||||||
|
+ 34.09 = 104.09 GiB) as parts, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*,
|
||||||
|
IQ4_XS) are single files. Fetch the parts, verify each against the Hub's LFS
|
||||||
|
sha256, concatenate in numeric order — that is how the deployed 111 GiB file
|
||||||
|
was assembled. Listing the Hub tree and filtering `endswith(".gguf")` makes
|
||||||
|
Q4_K_S look like it does not exist.
|
||||||
|
- **A fetch script must `-f` and verify, or it reports success on an access
|
||||||
|
error.** `curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body
|
||||||
|
as the "artifact": a gated repo yielded three 153-byte shards with identical
|
||||||
|
sha256 values and a cheerful `DOWNLOAD_DONE` in 20 seconds for 104 GiB. Use
|
||||||
|
`-f`/`--fail-with-body`, compare each file's byte count to the Hub's declared
|
||||||
|
size, and check the LFS sha256 — `scripts/download-frank-medium-gguf.sh`
|
||||||
|
already does all three; ad-hoc fetches tend not to. Gated repos (e.g.
|
||||||
|
`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, same 104.09 GiB Q4_K_S)
|
||||||
|
return 403 even with the host's HF token until that account accepts the gate.
|
||||||
|
- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup
|
||||||
|
campaign, four read-only scouts worked the same pages: one classified era pages as
|
||||||
|
stale *because* they document retired payloads (that is an era page's content), one
|
||||||
|
reported files that do not exist, one reported duplications that grep disproved
|
||||||
|
(`457.9 t/s` and the ping-RTT table are solo-held by `Ethernet Bandwidth Math`),
|
||||||
|
and one cited a line number past the end of the file it judged. Re-verify every
|
||||||
|
claimed duplicate and every claimed staleness against the file and today's fleet
|
||||||
|
state before editing; a scout yield buys no evidence.
|
||||||
|
|
||||||
## Runbook Index
|
## Runbook Index
|
||||||
|
|
||||||
|
|||||||
@@ -29,10 +29,10 @@
|
|||||||
# The scratch mirrors the production container contract exactly (serve-frank-
|
# The scratch mirrors the production container contract exactly (serve-frank-
|
||||||
# primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
|
# primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
|
||||||
# n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
|
# n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
|
||||||
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
|
# /dev/dri, the .rad ro, the tune cache mount. The ONLY differences: the
|
||||||
# differences: the container name (arbench, so prod's identity is untouched)
|
# container name (arbench, so prod's identity is untouched) and the prefix cache
|
||||||
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
|
# switched OFF (--prefix-cache-mib 0), so every prefill is cold — the tier is
|
||||||
# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
|
# host RAM now, there is no per-run disk directory to wipe.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
|
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
|
||||||
@@ -60,9 +60,8 @@ set -a; . "$ENV_FILE"; set +a
|
|||||||
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
|
# --- the prefix cache is switched off below, so there is no scratch KV tree ---
|
||||||
FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
|
# to wipe; the cold prefill comes from --prefix-cache-mib 0.
|
||||||
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
|
|
||||||
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
|
|
||||||
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
||||||
@@ -73,7 +72,6 @@ podman run -d --name "$SCRATCH" \
|
|||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device /dev/kfd --device /dev/dri \
|
--device /dev/kfd --device /dev/dri \
|
||||||
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
|
||||||
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
"$FRANK_ARCHON_IMAGE" \
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
@@ -87,12 +85,9 @@ podman run -d --name "$SCRATCH" \
|
|||||||
--placement "$FRANK_ARCHON_PLACEMENT" \
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
|
||||||
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
|
||||||
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
--prefix-cache-mib 0 \
|
||||||
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
--busy-prefill-chunk "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:-auto}" \
|
||||||
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
|
||||||
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
|
|||||||
@@ -1,293 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""OKF v0.1 conformance linter for a docs bundle (default serverAppConfigs-docs).
|
|
||||||
|
|
||||||
Checks the hard rules of the Open Knowledge Format spec plus the maintenance
|
|
||||||
guidance, so "conforms to OKF" is a test and not a claim:
|
|
||||||
|
|
||||||
FRONTMATTER every non-reserved .md has parseable YAML frontmatter with a
|
|
||||||
non-empty `type` from the spec's value set
|
|
||||||
TYPE the value is one of reference / howto / tutorial / explanation
|
|
||||||
DESC `description` is present, non-blank, not the title, <= 120 chars,
|
|
||||||
and does not restate a fleet quantity
|
|
||||||
TIMESTAMP `timestamp` present, ISO 8601, not in the future
|
|
||||||
LENGTH lines between 1 and 500 inclusive
|
|
||||||
INDEX-FM no index.md may carry frontmatter (index.md is reserved)
|
|
||||||
LOG-DATE log.md date headings are `## YYYY-MM-DD` or `## YYYY-MM-DD — Title`,
|
|
||||||
one entry per date, newest first
|
|
||||||
INDEX-COVER every index.md links each of its sibling pages; the root index
|
|
||||||
links every page in the bundle
|
|
||||||
INDEX-NUM an index table row restates a fleet quantity — a digit run of
|
|
||||||
three or more, a digits-plus-unit figure, a port, or a
|
|
||||||
multiplier. Names (Qwen3.6, RDNA4, RTX 3090, PICO-8, P4) are
|
|
||||||
not quantities and are allowed
|
|
||||||
CITATIONS informational: pages with factual claims and no Citations
|
|
||||||
section are counted, never failed
|
|
||||||
|
|
||||||
LENGTH lines between 1 and 500 inclusive. A page may opt out of the
|
|
||||||
ceiling with `okf-exempt: length` in its frontmatter: AGENTS.md
|
|
||||||
makes one page per endpoint permanent, and an endpoint page with
|
|
||||||
a full Eras table cannot split without creating a second owner
|
|
||||||
of the same shape (tests/docs_completeness.bats enforces that)
|
|
||||||
|
|
||||||
Exit 0 when there are no violations.
|
|
||||||
"""
|
|
||||||
import datetime
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
TYPES = {"reference", "howto", "tutorial", "explanation"}
|
|
||||||
RESERVED = {"index.md", "log.md", "AGENTS.md"}
|
|
||||||
|
|
||||||
# A quantity, as opposed to a name: a comma-grouped figure, a run of 3+ digits,
|
|
||||||
# a unit-bearing figure, a port (":7770"), or a multiplier ("2x", "3 *").
|
|
||||||
QUANTITY = re.compile(
|
|
||||||
r"\d{1,3}(?:,\d{3})+" # 262,144
|
|
||||||
r"|\d{3,}" # 200000, 7770, 3090
|
|
||||||
r"|\d+(?:\.\d+)?\s*(?:GiB|MiB|GB|MB|TB|tok/s|t/s|x\b|%)"
|
|
||||||
r"|[:×]\s*\d"
|
|
||||||
r"|\d+\s*[x×]\s*\d"
|
|
||||||
)
|
|
||||||
# Product / model / standard names that legitimately contain digits.
|
|
||||||
NAMES = re.compile(
|
|
||||||
r"Qwen[\d.]*[-\w]*|Gemma\s?\d[\w.]*|MiMo[-\w.\d.]*|Ornith[-\w.\d]*"
|
|
||||||
r"|Bonsai[-\w.\d]*|Ling[-\w.\d.]*|DeepSeek[-\w.\d]*|GLM[-\w.\d]*"
|
|
||||||
r"|PICO-\d+|RDNA\s?\d+|R\d{4}|RTX\s?\d+|XDNA\d|gfx\d+|CUDA\s?[\d.]+"
|
|
||||||
r"|\d+\s?GbE|Wi-?Fi\s?\d|v?\d+\.\d+(?:\.\d+)?|b\d+|P\d\b|A\d+\b"
|
|
||||||
r"|frank-\w*2026-\d\d-\d\d|2026-\d\d-\d\d",
|
|
||||||
re.I,
|
|
||||||
)
|
|
||||||
DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}")
|
|
||||||
FM_KEY = re.compile(r"^([A-Za-z][\w-]*):\s*(.*)$")
|
|
||||||
|
|
||||||
|
|
||||||
def split_frontmatter(text):
|
|
||||||
if not text.startswith("---"):
|
|
||||||
return None, text
|
|
||||||
end = text.find("\n---", 3)
|
|
||||||
if end < 0:
|
|
||||||
return None, text
|
|
||||||
body_start = text.find("\n", end + 1)
|
|
||||||
return text[3:end], text[body_start + 1 :] if body_start >= 0 else ""
|
|
||||||
|
|
||||||
|
|
||||||
def fields(raw):
|
|
||||||
out = {}
|
|
||||||
if not raw:
|
|
||||||
return out
|
|
||||||
key = None
|
|
||||||
for line in raw.splitlines():
|
|
||||||
if not line.strip():
|
|
||||||
continue
|
|
||||||
m = FM_KEY.match(line)
|
|
||||||
if m:
|
|
||||||
key, val = m.group(1), m.group(2).strip().strip("\"'")
|
|
||||||
out[key] = val
|
|
||||||
elif line.startswith(" ") and key:
|
|
||||||
out[key] = (out[key] + " " + line.strip()).strip()
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def strip_names(value):
|
|
||||||
return NAMES.sub(" ", value)
|
|
||||||
|
|
||||||
|
|
||||||
def check_file(path: Path, root: Path, length_exempt=frozenset()):
|
|
||||||
v = []
|
|
||||||
rel = str(path.relative_to(root))
|
|
||||||
text = path.read_text(errors="replace")
|
|
||||||
lines = text.splitlines()
|
|
||||||
name = path.name
|
|
||||||
raw, body = split_frontmatter(text)
|
|
||||||
fm = fields(raw)
|
|
||||||
|
|
||||||
if name not in RESERVED:
|
|
||||||
if raw is None:
|
|
||||||
v.append(("FRONTMATTER", rel, "no parseable frontmatter"))
|
|
||||||
else:
|
|
||||||
t = fm.get("type", "")
|
|
||||||
if not t:
|
|
||||||
v.append(("TYPE", rel, "missing type"))
|
|
||||||
elif t not in TYPES:
|
|
||||||
v.append(("TYPE", rel, f"type '{t}' not in {sorted(TYPES)}"))
|
|
||||||
d = fm.get("description", "")
|
|
||||||
if not d:
|
|
||||||
v.append(("DESC", rel, "missing description"))
|
|
||||||
else:
|
|
||||||
if len(d) > 120:
|
|
||||||
v.append(("DESC", rel, f"description is {len(d)} chars (max 120)"))
|
|
||||||
if d.strip("# ").lower() == fm.get("title", "\0").strip("# ").lower():
|
|
||||||
v.append(("DESC", rel, "description equals title"))
|
|
||||||
if QUANTITY.search(strip_names(d)):
|
|
||||||
v.append(("DESC", rel, "description restates a quantity"))
|
|
||||||
ts = fm.get("timestamp", "")
|
|
||||||
if not ts:
|
|
||||||
v.append(("TIMESTAMP", rel, "missing timestamp"))
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
stamp = datetime.date.fromisoformat(ts[:10])
|
|
||||||
if stamp > datetime.date.today():
|
|
||||||
v.append(("TIMESTAMP", rel, f"timestamp {ts} is in the future"))
|
|
||||||
except ValueError:
|
|
||||||
v.append(("TIMESTAMP", rel, f"unparseable timestamp {ts!r}"))
|
|
||||||
|
|
||||||
if name == "index.md" and raw is not None:
|
|
||||||
v.append(("INDEX-FM", rel, "index.md must not carry frontmatter"))
|
|
||||||
|
|
||||||
if name == "log.md":
|
|
||||||
dates = []
|
|
||||||
for line in lines:
|
|
||||||
m = re.match(r"^##\s+(\S.*?)\s*$", line)
|
|
||||||
if not m:
|
|
||||||
continue
|
|
||||||
head = m.group(1)
|
|
||||||
if not DATE_RE.match(head):
|
|
||||||
continue
|
|
||||||
dates.append(head[:10])
|
|
||||||
# OKF requires the heading to USE the date format; a title after an
|
|
||||||
# em-dash is allowed and costs zero lines, so `## 2026-08-17 — Why the
|
|
||||||
# pair was rebuilt` is conformant and `## 2026-08-17Rebuilt` is not.
|
|
||||||
if head != head[:10] and not re.match(r"^\d{4}-\d{2}-\d{2}\s+—\s+\S", head):
|
|
||||||
v.append(("LOG-DATE", rel,
|
|
||||||
f"date heading must be the date alone or `date — Title`: '## {head}'"))
|
|
||||||
if len(dates) != len(set(dates)):
|
|
||||||
v.append(("LOG-DATE", rel, "duplicate date heading"))
|
|
||||||
if dates != sorted(dates, reverse=True):
|
|
||||||
v.append(("LOG-DATE", rel, "date headings not newest first"))
|
|
||||||
|
|
||||||
if raw is not None and name not in RESERVED:
|
|
||||||
n = len(lines)
|
|
||||||
body_lines = len([ln for ln in body.splitlines() if ln.strip()])
|
|
||||||
opted_out = "length" in fm.get("okf-exempt", "").lower()
|
|
||||||
if n > 500:
|
|
||||||
if rel in length_exempt:
|
|
||||||
v.append(("LENGTH-OK", rel,
|
|
||||||
f"{n} lines — over the hint, pinned by the "
|
|
||||||
"one-page-per-endpoint law"))
|
|
||||||
elif opted_out:
|
|
||||||
v.append(("LENGTH-OK", rel,
|
|
||||||
f"{n} lines — over the hint, opted out by the page"))
|
|
||||||
else:
|
|
||||||
v.append(("LENGTH", rel, f"{n} lines (max 500)"))
|
|
||||||
elif body_lines < 15:
|
|
||||||
v.append(("LENGTH", rel, "stub: fewer than 15 body lines"))
|
|
||||||
|
|
||||||
# A second `type:` key inside the body means the page carries two copies of
|
|
||||||
# itself with a frontmatter block in the middle — the shape a naive
|
|
||||||
# frontmatter rewrite produces. It doubles the length, splits the wikilinks
|
|
||||||
# and makes the page's facts ambiguous, so it is a hard failure.
|
|
||||||
if raw is not None and name not in RESERVED:
|
|
||||||
dupes = [i + len((raw or "").splitlines()) + 2
|
|
||||||
for i, ln in enumerate(body.splitlines())
|
|
||||||
if re.match(r"^(type|title|description):", ln)]
|
|
||||||
if dupes:
|
|
||||||
v.append(("DOUBLED", rel,
|
|
||||||
f"frontmatter keys inside the body at line(s) "
|
|
||||||
f"{', '.join(str(d) for d in dupes[:5])} — the page "
|
|
||||||
"contains a duplicated copy of itself"))
|
|
||||||
|
|
||||||
return v
|
|
||||||
|
|
||||||
|
|
||||||
def index_rows(text):
|
|
||||||
rows = []
|
|
||||||
for line in text.splitlines():
|
|
||||||
if line.startswith("|") and "[[" in line:
|
|
||||||
rows.append(line)
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
def check_links(root: Path):
|
|
||||||
v = []
|
|
||||||
pages = [
|
|
||||||
p for p in sorted(root.rglob("*.md"))
|
|
||||||
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
|
|
||||||
]
|
|
||||||
for index in [p for p in pages if p.name == "index.md"]:
|
|
||||||
text = index.read_text(errors="replace")
|
|
||||||
links = set(re.findall(r"\[\[([^\]|#]+)", text))
|
|
||||||
if index.parent == root:
|
|
||||||
targets = [p for p in pages if p.name not in RESERVED]
|
|
||||||
else:
|
|
||||||
targets = [p for p in index.parent.glob("*.md") if p.name != "index.md"]
|
|
||||||
for target in targets:
|
|
||||||
if target.stem not in links and target.name not in links:
|
|
||||||
v.append(("INDEX-COVER", str(index.relative_to(root)),
|
|
||||||
f"does not link sibling {target.name}"))
|
|
||||||
for index in [p for p in pages if p.name == "index.md"]:
|
|
||||||
for row in index_rows(index.read_text(errors="replace")):
|
|
||||||
if QUANTITY.search(strip_names(row)):
|
|
||||||
v.append(("INDEX-NUM", str(index.relative_to(root)),
|
|
||||||
"row restates a quantity: " + row[:80]))
|
|
||||||
return v
|
|
||||||
|
|
||||||
|
|
||||||
def live_endpoints(root: Path):
|
|
||||||
"""Pages pinned by AGENTS.md's one-page-per-endpoint law.
|
|
||||||
|
|
||||||
Derived, never hardcoded — the same derivation as
|
|
||||||
tests/docs_completeness.bats: an endpoint is live when scripts/ holds both
|
|
||||||
its env and its unit. Those pages carry the Eras table for their whole
|
|
||||||
history, so the OKF 500-line split hint cannot apply to them without
|
|
||||||
creating a second owner of the same shape.
|
|
||||||
"""
|
|
||||||
scripts = root.parent / "scripts"
|
|
||||||
out = set()
|
|
||||||
if scripts.is_dir():
|
|
||||||
for envf in scripts.glob("frank-*.env"):
|
|
||||||
if (scripts / f"{envf.stem}.service").exists():
|
|
||||||
out.add(f"LLM/{envf.stem}.md")
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def retired_era_pages(root: Path):
|
|
||||||
"""Era pages of a retired endpoint: campaign ledgers, one owner each.
|
|
||||||
|
|
||||||
AGENTS.md lets a campaign ledger that exists nowhere else live on the page
|
|
||||||
that measured it, and a retired endpoint keeps exactly one page per era.
|
|
||||||
Splitting such a ledger to satisfy the 500-line hint would fragment one
|
|
||||||
campaign across two owners, so the tag opts the page out.
|
|
||||||
"""
|
|
||||||
out = set()
|
|
||||||
for path in root.rglob("*.md"):
|
|
||||||
s = str(path)
|
|
||||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
|
||||||
continue
|
|
||||||
raw, _ = split_frontmatter(path.read_text(errors="replace"))
|
|
||||||
if raw is not None and "retired-endpoint" in fields(raw).get("tags", ""):
|
|
||||||
out.add(str(path.relative_to(root)))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv):
|
|
||||||
root = Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
|
|
||||||
exempt = frozenset(live_endpoints(root)) | retired_era_pages(root)
|
|
||||||
violations = []
|
|
||||||
notes = []
|
|
||||||
citation_gaps = 0
|
|
||||||
pages = 0
|
|
||||||
for path in sorted(root.rglob("*.md")):
|
|
||||||
s = str(path)
|
|
||||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
|
||||||
continue
|
|
||||||
pages += 1
|
|
||||||
for code, rel, detail in check_file(path, root, exempt):
|
|
||||||
(notes if code == "LENGTH-OK" else violations).append((code, rel, detail))
|
|
||||||
text = path.read_text(errors="replace")
|
|
||||||
if len(text.splitlines()) > 40 and not re.search(
|
|
||||||
r"^##\s+(Citations|Sources)\b", text, re.M):
|
|
||||||
citation_gaps += 1
|
|
||||||
violations += check_links(root)
|
|
||||||
|
|
||||||
for code, rel, detail in violations:
|
|
||||||
print(f" {code:12s} {rel}: {detail}")
|
|
||||||
for code, rel, detail in notes:
|
|
||||||
print(f" {code:12s} {rel}: {detail}")
|
|
||||||
print(f"{pages} concept pages, {len(violations)} OKF violations"
|
|
||||||
f" ({citation_gaps} pages without a Citations section — soft;"
|
|
||||||
f" {len(notes)} length exemptions honoured)")
|
|
||||||
return 1 if violations else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main(sys.argv))
|
|
||||||
@@ -31,12 +31,7 @@ def strip_md(path: str) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def build_index(vault: pathlib.Path):
|
def build_index(vault: pathlib.Path):
|
||||||
# `.AGENTTRASH` is deleted-but-held state, not vault content: it must not be
|
pages = sorted(vault.rglob("*.md"))
|
||||||
# scanned (its relative links resolve against the trash dir, not the vault)
|
|
||||||
# and must not answer as a link target — a live page pointing into the trash
|
|
||||||
# is an unresolved link, not a resolved one.
|
|
||||||
pages = sorted(p for p in vault.rglob("*.md")
|
|
||||||
if ".AGENTTRASH" not in str(p))
|
|
||||||
stems: dict[str, list[pathlib.Path]] = {}
|
stems: dict[str, list[pathlib.Path]] = {}
|
||||||
for page in pages:
|
for page in pages:
|
||||||
stems.setdefault(page.stem.casefold(), []).append(page)
|
stems.setdefault(page.stem.casefold(), []).append(page)
|
||||||
|
|||||||
+56
-25
@@ -7,10 +7,15 @@
|
|||||||
# only the reason a reader editing THIS line cannot recover from the page.
|
# only the reason a reader editing THIS line cannot recover from the page.
|
||||||
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
||||||
# the engine shape.
|
# the engine shape.
|
||||||
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
|
||||||
# 1.0.13 is pinned by digest on purpose: 1.2.0 is a DEAD END on this payload
|
# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
|
||||||
# (acceptance 0.7123 vs 0.7549 — the page's Dead ends section names the suspect).
|
# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
|
||||||
# The .rad file mounted under /models; /data holds the prefix-cache disk tier.
|
# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
|
||||||
|
# now split by the engine's budget resolver — see below), and the prefix cache
|
||||||
|
# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
|
||||||
|
# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
|
||||||
|
# suspect is untangled (the page's Dead ends section names it).
|
||||||
|
# The .rad file mounts under /models; the tune cache mounts separately below.
|
||||||
# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
|
# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
|
||||||
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
||||||
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
|
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
|
||||||
@@ -52,21 +57,30 @@ FRANK_ARCHON_MAX_SEQS=3
|
|||||||
# sequence.
|
# sequence.
|
||||||
FRANK_ARCHON_MAX_MODEL_LEN=200000
|
FRANK_ARCHON_MAX_MODEL_LEN=200000
|
||||||
FRANK_ARCHON_PLACEMENT=expert_tiered
|
FRANK_ARCHON_PLACEMENT=expert_tiered
|
||||||
# Pinned host pool for expert rows that do not fit VRAM. It moves WITH
|
# Pinned host pool for expert rows that do not fit VRAM. Under the budget
|
||||||
# VRAM_WEIGHTS: the pair and this pool serve the per-rank resident set together,
|
# resolver this is ALSO the lever that decides which experts stay on the card:
|
||||||
# so cutting the slab spills here — under-sizing it is a boot failure (36,864
|
# the card's leftover goes to the expert plane first and the KV cache takes the
|
||||||
|
# rest, so growing the pool moves experts off the card and shrinking it spills
|
||||||
|
# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
|
||||||
# refused a boot). The ledger's coverage line is the referee, not this file.
|
# refused a boot). The ledger's coverage line is the referee, not this file.
|
||||||
FRANK_ARCHON_HOST_POOL_MIB=44928
|
# Sized so the cache clears the 3 x MAX_MODEL_LEN contract: the 1.8.1 boot at
|
||||||
|
# 44,928 resolved the cache to 556,348 tokens — 2.8 sessions — and three
|
||||||
|
# concurrent full-height requests came back 400 in 0.0 s. Tokens scale with the
|
||||||
|
# GiB the cache wins, so +640 MiB a rank buys ~44,000 tokens of ceiling with
|
||||||
|
# ~2.5% off the expert slab.
|
||||||
|
FRANK_ARCHON_HOST_POOL_MIB=45568
|
||||||
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
||||||
# floor is built from it.
|
# floor is built from it.
|
||||||
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
# ONE budget per card: VRAM_WEIGHTS + VRAM_KV <= 30,969 MiB, the ceiling the
|
# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
|
||||||
# engine's own budget table prints (derivation and the slab/pool price: the
|
# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
|
||||||
# page's VRAM laws). Idle KV VRAM is ALSO the expert cache's spillover, so a
|
# puts the static (non-expert) weights off the top, then divides the rest between
|
||||||
# bigger pool pays twice — which is why the pool sits at the contract and the
|
# the expert plane and the KV cache — experts first when the cache has no idle
|
||||||
# slab takes the rest.
|
# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
|
||||||
FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
|
# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
|
||||||
FRANK_ARCHON_VRAM_KV_MIB=5120
|
# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
|
||||||
|
# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
|
||||||
|
# it after every resize instead of asserting it here.
|
||||||
# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
|
# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
|
||||||
# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
|
# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
|
||||||
# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
|
# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
|
||||||
@@ -75,20 +89,37 @@ FRANK_ARCHON_KV_DTYPE=fp8
|
|||||||
# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
|
# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
|
||||||
FRANK_ARCHON_SPEC_TOKENS=2
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
# Prefix cache tiers. The host tier is sized in ENTRIES and one entry tracks the
|
# Step sharing. A prefill chunk that shares a step with decoding sequences holds
|
||||||
# cache dtype (58.50 KiB at fp8), so a dtype change re-sizes the parked set
|
# the device for the chunk's whole time, and every decode row in that step waits
|
||||||
# without touching these numbers.
|
# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
|
||||||
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
# `auto` leaves each decode row behind the chunk a row plus its draft window and
|
||||||
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
|
||||||
|
# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
|
||||||
|
# chunk took the step and admission never ran. Interleave inserts N decode-only
|
||||||
|
# steps after a chunk step taken beside decoders: it protects their cadence and
|
||||||
|
# holds a waiting request out of those steps, so it stays 0 until the post-migration
|
||||||
|
# decode-vs-prefill buckets say it is wanted.
|
||||||
|
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
|
||||||
|
FRANK_ARCHON_PREFILL_INTERLEAVE=0
|
||||||
|
# Prefix cache: pinned host memory only. The disk tier was removed outright
|
||||||
|
# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
|
||||||
|
# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
|
||||||
|
# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
|
||||||
|
# this change: a drop is extra prefill, and prefill is exactly the traffic that
|
||||||
|
# costs decode its cadence.
|
||||||
|
FRANK_ARCHON_PREFIX_CACHE_MIB=8192
|
||||||
# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
|
# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
|
||||||
# read. `ram` REFUSES the start if memlock is too small — the unit's
|
# read. `ram` REFUSES the start if memlock is too small — the unit's
|
||||||
# LimitMEMLOCK=infinity is the other half of this law.
|
# LimitMEMLOCK=infinity is the other half of this law.
|
||||||
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||||
# TTM preflight. The pinned ledger (~147 GiB) rides TTM's shared pages, and the
|
# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
|
||||||
# kernel RESETS pages_limit to half of RAM on every boot; below the plan the
|
# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
|
||||||
# load dies mid-pin with an anonymous allocator error. ttm-preflight.sh raises
|
# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
|
||||||
# the knob itself before refusing by name, so recovery is a boot, not a visit.
|
# itself before refusing by name, so recovery is a boot, not a visit.
|
||||||
FRANK_ARCHON_TTM_MIN_MIB=150000
|
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
|
||||||
|
# per the bats ledger test; at the 45,568 pool that computes to 150,200 MiB, so
|
||||||
|
# the floor sits just above it.
|
||||||
|
FRANK_ARCHON_TTM_MIN_MIB=152000
|
||||||
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
||||||
# this unquoted on purpose, so keep multi-flag values as one string.
|
# this unquoted on purpose, so keep multi-flag values as one string.
|
||||||
|
|||||||
@@ -1,206 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Regenerate serverAppConfigs-docs/index.md from page frontmatter.
|
|
||||||
|
|
||||||
The root index is a locator (AGENTS.md -> *Documentation Policy*): one row per
|
|
||||||
page, one clause, no numbers. Hand-maintaining it is how rows go missing, and
|
|
||||||
the docs seam fails when a page is not linked from it, so it is generated:
|
|
||||||
|
|
||||||
* preamble everything above the first `## ` in the existing file is kept
|
|
||||||
verbatim (that is the curated statement of the vault's law)
|
|
||||||
* section from the page's directory, overridden by its frontmatter tags
|
|
||||||
(`endpoint` / `retired-endpoint` split the LLM pages)
|
|
||||||
* row text the page's frontmatter `description`, verbatim
|
|
||||||
|
|
||||||
A page with no description is a hard error rather than an empty cell, so the
|
|
||||||
generator cannot silently produce a silent index. Sections with no pages are
|
|
||||||
skipped. Deterministic: the same vault always yields the same bytes.
|
|
||||||
|
|
||||||
Sub-directory indices are hand-grouped (they carry curated prose), so they are
|
|
||||||
synced rather than regenerated: dead rows dropped, row text refreshed from the
|
|
||||||
page's description, uncovered siblings appended.
|
|
||||||
"""
|
|
||||||
import pathlib
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
|
|
||||||
RESERVED = {"index.md", "log.md"}
|
|
||||||
SECTIONS = [
|
|
||||||
("Inference endpoints", "endpoint"),
|
|
||||||
("Retired endpoints", "retired-endpoint"),
|
|
||||||
]
|
|
||||||
DIRECTORY_SECTIONS = [
|
|
||||||
("Dankclaw", "Dankclaw services"),
|
|
||||||
("Dankserver", "Dankserver services"),
|
|
||||||
("FrankBook-Pro", "FrankBook-Pro"),
|
|
||||||
("Desktop", "Workstations"),
|
|
||||||
("Infrastructure", "Cross-host infrastructure"),
|
|
||||||
("Architecture", "Repo architecture and policy"),
|
|
||||||
("Operations", "Operations and incidents"),
|
|
||||||
("Agent", "Agent / harness"),
|
|
||||||
("Research", "Research and methodology"),
|
|
||||||
]
|
|
||||||
PIPELINE_SECTION = "Pipeline Parallelism (series)"
|
|
||||||
LLM_FALLBACK = "LLM technique and hardware references"
|
|
||||||
CHANGE_SECTION = "Change history"
|
|
||||||
ORDER = [t for t, _ in SECTIONS] + [t for _, t in DIRECTORY_SECTIONS] + \
|
|
||||||
[PIPELINE_SECTION, LLM_FALLBACK, CHANGE_SECTION]
|
|
||||||
|
|
||||||
|
|
||||||
def frontmatter_fields(text):
|
|
||||||
if not text.startswith("---"):
|
|
||||||
return {}
|
|
||||||
end = text.find("\n---", 3)
|
|
||||||
if end == -1:
|
|
||||||
return {}
|
|
||||||
raw = text[3:end]
|
|
||||||
fields = {}
|
|
||||||
for key in ("description", "type"):
|
|
||||||
m = re.search(rf"^{key}:\s*(.*)$", raw, re.M)
|
|
||||||
if m:
|
|
||||||
fields[key] = m.group(1).strip().strip("\"'")
|
|
||||||
tags = []
|
|
||||||
m = re.search(r"^tags:\s*\[(.*?)\]$", raw, re.M)
|
|
||||||
if m:
|
|
||||||
tags = [t.strip().strip("\"'") for t in m.group(1).split(",") if t.strip()]
|
|
||||||
elif re.search(r"^tags:\s*$", raw, re.M):
|
|
||||||
block = re.search(r"^tags:\s*\n((?:\s+-\s+.*\n?)+)", raw, re.M)
|
|
||||||
if block:
|
|
||||||
tags = [l.strip().lstrip("-").strip().strip("\"'")
|
|
||||||
for l in block.group(1).splitlines() if l.strip()]
|
|
||||||
fields["tags"] = tags
|
|
||||||
return fields
|
|
||||||
|
|
||||||
|
|
||||||
def section_for(page, vault, fields):
|
|
||||||
rel = page.relative_to(vault)
|
|
||||||
if len(rel.parts) == 1:
|
|
||||||
return CHANGE_SECTION if page.name == "log.md" else "Uncategorised"
|
|
||||||
top = rel.parts[0]
|
|
||||||
if top == "LLM":
|
|
||||||
if PIPELINE_SECTION_DIR in rel.parts:
|
|
||||||
return PIPELINE_SECTION
|
|
||||||
for title, tag in SECTIONS:
|
|
||||||
if tag in fields.get("tags", []):
|
|
||||||
return title
|
|
||||||
return LLM_FALLBACK
|
|
||||||
for prefix, title in DIRECTORY_SECTIONS:
|
|
||||||
if top == prefix:
|
|
||||||
return title
|
|
||||||
return "Uncategorised"
|
|
||||||
|
|
||||||
|
|
||||||
ROW = re.compile(r"^(\| )\[\[([^\]|]+)\]\]( \|)( .*?)( \|$)")
|
|
||||||
BULLET = re.compile(r"^(- )\[\[([^\]]+)\]\]( — )(.*?)$")
|
|
||||||
|
|
||||||
|
|
||||||
def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
|
|
||||||
"""Keep a hand-grouped sub-index honest without erasing its grouping.
|
|
||||||
|
|
||||||
Rows are locators, so their text is the linked page's own description; a
|
|
||||||
row whose page is gone is a dead promise and is dropped; a direct sibling
|
|
||||||
with no row is appended under `## Pages` (that is what the link-integrity
|
|
||||||
seam checks). Curated headings and prose are untouched.
|
|
||||||
"""
|
|
||||||
here = index.parent
|
|
||||||
known = {p.stem for p in here.rglob("*.md")
|
|
||||||
if ".AGENTTRASH" not in str(p) and p.name != "index.md"}
|
|
||||||
lines = index.read_text().split("\n")
|
|
||||||
out, dropped, refreshed = [], 0, 0
|
|
||||||
for line in lines:
|
|
||||||
for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
|
|
||||||
m = pat.match(line)
|
|
||||||
if not m:
|
|
||||||
continue
|
|
||||||
stem = m.group(2).strip()
|
|
||||||
if stem not in known:
|
|
||||||
dropped += 1
|
|
||||||
line = None
|
|
||||||
break
|
|
||||||
cand = next((p for p in here.rglob(f"{stem}.md")
|
|
||||||
if ".AGENTTRASH" not in str(p)), None)
|
|
||||||
desc = frontmatter_fields(cand.read_text(errors="replace")).get("description") \
|
|
||||||
if cand else None
|
|
||||||
if desc:
|
|
||||||
current = m.group(4).strip()
|
|
||||||
if current != desc:
|
|
||||||
line = wrap.format(stem, " " if pat is ROW else "", desc) \
|
|
||||||
if pat is ROW else wrap.format(stem, desc)
|
|
||||||
refreshed += 1
|
|
||||||
break
|
|
||||||
if line is not None:
|
|
||||||
out.append(line)
|
|
||||||
text = "\n".join(out)
|
|
||||||
# Presence must be the wikilink itself: a bare stem is a substring of other
|
|
||||||
# page names (`frank-medium` lives inside `frank-medium-rocmfpx`), and a
|
|
||||||
# sibling matched that way silently never gets a row.
|
|
||||||
missing = sorted(
|
|
||||||
(p.stem, frontmatter_fields(p.read_text(errors="replace")).get("description", ""))
|
|
||||||
for p in here.glob("*.md")
|
|
||||||
if p.name != "index.md" and f"[[{p.stem}]]" not in text
|
|
||||||
)
|
|
||||||
if missing:
|
|
||||||
block = ["", "## Pages", "", "| Page | Owns |", "|---|---|"]
|
|
||||||
block += [f"| [[{stem}]] | {desc} |" for stem, desc in missing]
|
|
||||||
text = text.rstrip("\n") + "\n" + "\n".join(block) + "\n"
|
|
||||||
if text != index.read_text():
|
|
||||||
index.write_text(text)
|
|
||||||
return dropped, refreshed, len(missing)
|
|
||||||
|
|
||||||
|
|
||||||
PIPELINE_SECTION_DIR = "Pipeline Parallelism"
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv):
|
|
||||||
vault = pathlib.Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
|
|
||||||
index = vault / "index.md"
|
|
||||||
if not index.is_file():
|
|
||||||
print(f"missing {index}")
|
|
||||||
return 2
|
|
||||||
existing = index.read_text()
|
|
||||||
preamble = existing.split("\n## ", 1)[0].rstrip("\n")
|
|
||||||
|
|
||||||
pages = [p for p in vault.rglob("*.md")
|
|
||||||
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
|
|
||||||
and p.name != "index.md" and p != index]
|
|
||||||
buckets, errors = {}, []
|
|
||||||
for page in pages:
|
|
||||||
fields = frontmatter_fields(page.read_text(errors="replace"))
|
|
||||||
if page.name != "log.md":
|
|
||||||
desc = fields.get("description")
|
|
||||||
if not desc:
|
|
||||||
errors.append(str(page.relative_to(vault)))
|
|
||||||
continue
|
|
||||||
else:
|
|
||||||
desc = "Chronological incident and law log for the whole bundle."
|
|
||||||
buckets.setdefault(section_for(page, vault, fields), []).append((page.stem, desc))
|
|
||||||
if errors:
|
|
||||||
print("pages with no description (fix the frontmatter, not the index):")
|
|
||||||
for e in sorted(errors):
|
|
||||||
print(" " + e)
|
|
||||||
return 1
|
|
||||||
|
|
||||||
out = [preamble, ""]
|
|
||||||
for title in ORDER + sorted(set(buckets) - set(ORDER)):
|
|
||||||
rows = sorted(buckets.get(title, []), key=lambda r: r[0].lower())
|
|
||||||
if not rows:
|
|
||||||
continue
|
|
||||||
out += [f"## {title}", "", "| Page | Owns |", "|---|---|"]
|
|
||||||
out += [f"| [[{stem}]] | {desc} |" for stem, desc in rows]
|
|
||||||
out.append("")
|
|
||||||
out.append("<!-- generated by scripts/gen-vault-index.py; edit page frontmatter instead -->")
|
|
||||||
index.write_text("\n".join(out) + "\n")
|
|
||||||
touched = 0
|
|
||||||
for sub in sorted(vault.rglob("index.md")):
|
|
||||||
if sub == index:
|
|
||||||
continue
|
|
||||||
dropped, refreshed, added = sync_subindex(sub, vault)
|
|
||||||
if dropped or refreshed or added:
|
|
||||||
touched += 1
|
|
||||||
print(f" {sub}: -{dropped} dead rows, {refreshed} refreshed, +{added} added")
|
|
||||||
print(f"{index}: {sum(len(v) for v in buckets.values())} rows in "
|
|
||||||
f"{len([t for t in ORDER if buckets.get(t)])} sections, {touched} sub-indices synced")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main(sys.argv))
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Port census: prove a merge/collapse lost no fact.
|
|
||||||
|
|
||||||
For every line that existed in the OLD set and not in the NEW set, check that
|
|
||||||
each of its significant tokens (numbers, long identifiers, paths) still occurs
|
|
||||||
somewhere in the NEW corpus. A line whose every token survives is a reworded
|
|
||||||
line; a line with an unheld token is a candidate fact loss and gets printed.
|
|
||||||
|
|
||||||
port-census.py <new-glob-dir> <old-file> [<old-file> ...]
|
|
||||||
|
|
||||||
Old files may be `git-rev:path` (read from git) or a path (e.g. a trash copy).
|
|
||||||
"""
|
|
||||||
import pathlib
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
def toks(line):
|
|
||||||
out = []
|
|
||||||
for n in re.findall(r"\d[\d,]*(?:\.\d+)?", line):
|
|
||||||
if len(n.replace(",", "").split(".")[0]) >= 3 or (
|
|
||||||
"." in n and len(n.split(".")[-1]) >= 2 and len(n.replace(",", "")) >= 4
|
|
||||||
):
|
|
||||||
out.append(n)
|
|
||||||
# Identifiers, flags, paths and backticked spans are the facts worth
|
|
||||||
# tracking; ordinary prose words are not — they reword on every rewrite.
|
|
||||||
# A hyphen alone is not evidence of an identifier (`long-standing`), so a
|
|
||||||
# dashed word counts only with two hyphens (`frank-tts-engine`) or with a
|
|
||||||
# digit, dot, slash, colon or `--` inside it.
|
|
||||||
for w in re.findall(r"[A-Za-z][\w./:-]{7,}", line):
|
|
||||||
core = w.strip(".,;:!?\"')]")
|
|
||||||
if (core.startswith("--") or core.isupper()
|
|
||||||
or re.search(r"[\d_/:.]", core) or core.count("-") >= 2):
|
|
||||||
out.append(core)
|
|
||||||
out += [s.strip() for s in re.findall(r"`([^`]+)`", line) if len(s) > 3]
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def held(tok, corpus):
|
|
||||||
t = tok.replace(",", "")
|
|
||||||
alt = rf"\b{re.escape(t)}\b"
|
|
||||||
for c in corpus:
|
|
||||||
if tok in c or t in c:
|
|
||||||
return True
|
|
||||||
if re.search(alt, c.replace(",", "")):
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv):
|
|
||||||
new_dir = pathlib.Path(argv[1])
|
|
||||||
|
|
||||||
def corpus_excluding(exclude):
|
|
||||||
"""Every vault page except the one under test.
|
|
||||||
|
|
||||||
The corpus is what must still hold a fact AFTER the file is gone, so a
|
|
||||||
file that lives inside the scanned directory has to be removed from it —
|
|
||||||
otherwise every token is "held" by the file itself and the census proves
|
|
||||||
nothing (it reports 0 losses for every page, which is the bug this
|
|
||||||
guards).
|
|
||||||
"""
|
|
||||||
out = []
|
|
||||||
for p in new_dir.rglob("*.md"):
|
|
||||||
s = str(p)
|
|
||||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
|
||||||
continue
|
|
||||||
if exclude is not None and p.resolve() == exclude:
|
|
||||||
continue
|
|
||||||
out.append(p.read_text(errors="replace"))
|
|
||||||
return out
|
|
||||||
|
|
||||||
lost = checked = 0
|
|
||||||
for spec in argv[2:]:
|
|
||||||
if spec.startswith("git-"):
|
|
||||||
ref, path = spec[4:].split(":", 1)
|
|
||||||
old = subprocess.run(["git", "show", f"{ref}:{path}"],
|
|
||||||
capture_output=True, text=True).stdout
|
|
||||||
label = f"{ref}:{path}"
|
|
||||||
exclude = None
|
|
||||||
else:
|
|
||||||
old = pathlib.Path(spec).read_text(errors="replace")
|
|
||||||
label = spec
|
|
||||||
exclude = pathlib.Path(spec).resolve()
|
|
||||||
blob = "\n".join(corpus_excluding(exclude))
|
|
||||||
for line in old.splitlines():
|
|
||||||
s = line.strip().strip("->*|# ").strip()
|
|
||||||
if len(s) < 30 or s.startswith("```"):
|
|
||||||
continue
|
|
||||||
checked += 1
|
|
||||||
missing = [t for t in toks(s) if not held(t, [blob])]
|
|
||||||
if missing:
|
|
||||||
lost += 1
|
|
||||||
print(f" {label}\n L: {s[:150]}\n unheld: {', '.join(missing[:6])}")
|
|
||||||
print(f"{checked} old lines checked, {lost} with unheld tokens")
|
|
||||||
return 1 if lost else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main(sys.argv))
|
|
||||||
@@ -15,12 +15,13 @@
|
|||||||
# log states every budget before anything is allocated (weights vs KV
|
# log states every budget before anything is allocated (weights vs KV
|
||||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||||
# the P2P answer per pair) — read it after every resize.
|
# the P2P answer per pair) — read it after every resize.
|
||||||
# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
|
# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
|
||||||
# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
|
# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
|
||||||
# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
|
# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
|
||||||
# own budget table makes claimable, spent on the slab because prefill is
|
# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
|
||||||
# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
|
# and the budget resolver derives it from headroom, the cache bound and the host
|
||||||
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
|
# pool, so the slab size and the pool's token count are LEDGER lines: after every
|
||||||
|
# boot read them out of the log and check the pool still backs 3 x 200,000.
|
||||||
#
|
#
|
||||||
# The root unit (scripts/frank-archon.service, User=dank,
|
# The root unit (scripts/frank-archon.service, User=dank,
|
||||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
||||||
@@ -56,13 +57,11 @@ set +a
|
|||||||
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
|
||||||
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
|
: "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:?FRANK_ARCHON_BUSY_PREFILL_CHUNK must be set (auto|off|N)}"
|
||||||
: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
|
|
||||||
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
|
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
|
||||||
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
|
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
|
||||||
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
||||||
: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
|
: "${FRANK_ARCHON_PREFIX_CACHE_MIB:?FRANK_ARCHON_PREFIX_CACHE_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
|
||||||
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
||||||
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
|
: "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
|
||||||
@@ -103,19 +102,12 @@ if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
|
|
||||||
# The engine's create is NOT recursive: with only the state dir mounted, its
|
|
||||||
# "cannot create /data/kvcache/frank-archon: No such file or directory"
|
|
||||||
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
|
|
||||||
# the full path on the host so the tier opens on first boot.
|
|
||||||
|
|
||||||
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
||||||
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
||||||
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
||||||
# the cache must ride a host mount or it dies with the --rm container
|
# the cache must ride a host mount or it dies with the --rm container
|
||||||
# that wrote it. The dir is pre-created for the same reason as kvcache
|
# that wrote it. The engine's create is not recursive, so the dir is pre-created.
|
||||||
# (the engine's create is not recursive).
|
|
||||||
|
|
||||||
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
||||||
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
|
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
|
||||||
@@ -148,6 +140,15 @@ done
|
|||||||
LOG=/home/dank/logs/frank_archon.log
|
LOG=/home/dank/logs/frank_archon.log
|
||||||
mkdir -p "$(dirname "$LOG")"
|
mkdir -p "$(dirname "$LOG")"
|
||||||
|
|
||||||
|
# Interleave is opt-in: 0 is the engine's default and passing it anyway would put
|
||||||
|
# a flag in argv that changes nothing but the record. The sharing cap is always
|
||||||
|
# explicit because `auto` vs `off` is the difference between decode waiting one
|
||||||
|
# chunk and waiting the whole step.
|
||||||
|
PREFILL_ARGS=()
|
||||||
|
if [ "${FRANK_ARCHON_PREFILL_INTERLEAVE:-0}" -gt 0 ]; then
|
||||||
|
PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")
|
||||||
|
fi
|
||||||
|
|
||||||
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
||||||
# --network host (the server binds the host's addresses directly, no NAT),
|
# --network host (the server binds the host's addresses directly, no NAT),
|
||||||
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
|
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
|
||||||
@@ -156,8 +157,9 @@ mkdir -p "$(dirname "$LOG")"
|
|||||||
# makes for host memory), memlock unbounded (the pinned ledger — legal
|
# makes for host memory), memlock unbounded (the pinned ledger — legal
|
||||||
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
|
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
|
||||||
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
||||||
# The template rides a single-file mount; the .rad dir is ro; /data holds
|
# The template rides a single-file mount and the .rad dir is ro. The prefix cache
|
||||||
# the prefix-cache disk tier.
|
# has no disk tier any more, so the only host state this container writes is the
|
||||||
|
# rad-tune cache under /opt/radiance/share/radiance/tune.
|
||||||
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
||||||
--network host \
|
--network host \
|
||||||
--init \
|
--init \
|
||||||
@@ -165,7 +167,6 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
|||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device /dev/kfd --device /dev/dri \
|
--device /dev/kfd --device /dev/dri \
|
||||||
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
|
||||||
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
"$FRANK_ARCHON_IMAGE" \
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
@@ -179,15 +180,13 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
|||||||
--placement "$FRANK_ARCHON_PLACEMENT" \
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
|
||||||
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
|
||||||
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
--prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB" \
|
||||||
--prefix-cache-dir /data/kvcache/frank-archon \
|
|
||||||
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
|
||||||
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
|
--busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK" \
|
||||||
|
"${PREFILL_ARGS[@]}" \
|
||||||
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Agent Docs and Edits
|
title: Agent Docs and Edits
|
||||||
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
|
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
|
||||||
tags: [agents, docs, markdown, hashline, edit-recovery]
|
tags: [agents, docs, markdown, hashline, edit-recovery]
|
||||||
timestamp: 2026-10-18T00:00:00Z
|
timestamp: 2026-10-18T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Command Center Dotfiles
|
title: Command Center Dotfiles
|
||||||
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
|
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
|
||||||
tags: [dotfiles, omp, deploy, rsync, blankpad]
|
tags: [dotfiles, omp, deploy, rsync, blankpad]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: OMP Harness
|
title: OMP Harness
|
||||||
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
|
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
|
||||||
tags: [omp, harness, compaction, models-yml, context, samplers]
|
tags: [omp, harness, compaction, models-yml, context, samplers]
|
||||||
timestamp: 2026-10-18T00:00:00Z
|
timestamp: 2026-10-18T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,12 +1,23 @@
|
|||||||
|
---
|
||||||
|
type: index
|
||||||
|
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
|
||||||
|
tags: [agent, index]
|
||||||
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
|
---
|
||||||
|
|
||||||
# Agent
|
# Agent
|
||||||
|
|
||||||
Pages about the agent's own tooling and context — how omp is configured, how
|
Pages about the agent's own tooling and context — how omp is configured, how
|
||||||
docs get edited safely, and how the agent's home is deployed.
|
docs get edited safely, and how the agent's home is deployed.
|
||||||
|
|
||||||
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
|
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
|
||||||
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
|
`compat.extraBody`, preamble and skill-library audits
|
||||||
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
|
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
|
||||||
|
hashline edit recovery
|
||||||
|
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
|
||||||
|
transport, override packages
|
||||||
|
|
||||||
## Related
|
## Related
|
||||||
|
|
||||||
|
- [[Testing Policy]] — the seams these pages' checks ride
|
||||||
- [LLM](../LLM/index.md) — the fleet these agents operate
|
- [LLM](../LLM/index.md) — the fleet these agents operate
|
||||||
@@ -1,7 +1,8 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Container Network Namespaces
|
title: Container Network Namespaces
|
||||||
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
|
description: Which local services are addressable from which Podman network namespace on
|
||||||
|
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
|
||||||
tags:
|
tags:
|
||||||
- networking
|
- networking
|
||||||
- podman
|
- podman
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Deploy State Triage
|
title: Deploy State Triage
|
||||||
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
|
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
|
||||||
tags: [deployment, deploy-state, systemd, podman, triage]
|
tags: [deployment, deploy-state, systemd, podman, triage]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
type: howto
|
type: guide
|
||||||
title: Deployment Flow
|
title: Deployment Flow
|
||||||
description: Service-scoped Git deployment for the two homelab hosts.
|
description: Service-scoped Git deployment for the two homelab hosts.
|
||||||
tags:
|
tags:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
|
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
|
||||||
tags:
|
tags:
|
||||||
- architecture
|
- architecture
|
||||||
- e2e
|
- e2e
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Quadlet Lifecycle
|
title: Quadlet Lifecycle
|
||||||
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
|
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
|
||||||
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
|
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Stale Signal Audit
|
title: Stale Signal Audit
|
||||||
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
|
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
|
||||||
tags: [testing, observability, audit, signals, gates]
|
tags: [testing, observability, audit, signals, gates]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: howto
|
type: howto
|
||||||
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
|
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- comfyui
|
- comfyui
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
|
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- tts
|
- tts
|
||||||
|
|||||||
@@ -1,13 +1,11 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
|
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
|
||||||
tags:
|
tags:
|
||||||
- endpoint
|
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- tts
|
- tts
|
||||||
- voice
|
- voice
|
||||||
- quadlet
|
- quadlet
|
||||||
timestamp: 2026-10-19
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Frank TTS
|
# Frank TTS
|
||||||
@@ -204,4 +202,3 @@ harness bundle and measured the day of the flip:
|
|||||||
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
||||||
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
||||||
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
|
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
|
||||||
---
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: Gitea
|
title: Gitea
|
||||||
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
|
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
|
||||||
tags:
|
tags:
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: Hermes-WebUI (Hermex server)
|
title: Hermes-WebUI (Hermex server)
|
||||||
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
|
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
|
||||||
tags:
|
tags:
|
||||||
- hermes
|
- hermes
|
||||||
- hermes-webui
|
- hermes-webui
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Marreta on Dankclaw
|
title: Marreta on Dankclaw
|
||||||
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
|
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- marreta
|
- marreta
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
|
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- open-notebook
|
- open-notebook
|
||||||
@@ -13,7 +13,7 @@ timestamp: 2026-10-18
|
|||||||
|
|
||||||
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
||||||
|
|
||||||
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Re-censused 2026-10-19 and unchanged: no `*notebook*` unit in `systemctl --user list-unit-files`, nothing installed under `~/.config/containers/systemd/` by that name, no container in `podman ps` — the repo's three quadlet files have never been installed on the host. Deploying it is a deliberate next step, not a pending one.
|
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
|
||||||
|
|
||||||
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
|
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- seerr
|
- seerr
|
||||||
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
|
|||||||
authentik's own user UI instead of Seerr. Both backfilled live: the three
|
authentik's own user UI instead of Seerr. Both backfilled live: the three
|
||||||
standard flows copied from a known-good provider, plus an application
|
standard flows copied from a known-good provider, plus an application
|
||||||
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
|
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
|
||||||
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
|
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
|
||||||
two provisioning rules this incident established.
|
provisioning rules this incident established.
|
||||||
|
|
||||||
## *arr wiring
|
## *arr wiring
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: Dankclaw Swap Configuration
|
title: Dankclaw Swap Configuration
|
||||||
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
|
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- host
|
- host
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
|
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- frank-tts
|
- frank-tts
|
||||||
|
|||||||
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
|
|||||||
|
|
||||||
## Security Advisories
|
## Security Advisories
|
||||||
|
|
||||||
| Version | Advisory | CVE | Issue |
|
| Version | CVE | Issue |
|
||||||
|---------|----------|-----|-------|
|
|---------|-----|-------|
|
||||||
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
|
||||||
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
|
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
|
||||||
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
|
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
|
||||||
|
|
||||||
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
|
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
|
||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
@@ -174,10 +174,3 @@ Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) ma
|
|||||||
- [[Dankserver Stack]]
|
- [[Dankserver Stack]]
|
||||||
- [[Caddy]] — TLS termination and authentication in front
|
- [[Caddy]] — TLS termination and authentication in front
|
||||||
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
|
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
|
||||||
|
|
||||||
## Citations
|
|
||||||
|
|
||||||
- GitHub Security Advisories, `9001/copyparty` —
|
|
||||||
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
|
|
||||||
each row's patched version read from `first_patched_version` in the
|
|
||||||
`https://api.github.com/advisories/<GHSA-id>` response.
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Service inventory and routing overview for the media and library host."
|
description: "Dankserver (192.168.0.235) runs the media and library stack."
|
||||||
tags:
|
tags:
|
||||||
- dankserver
|
- dankserver
|
||||||
timestamp: 2026-09-03
|
timestamp: 2026-09-03
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Jellyfin
|
title: Jellyfin
|
||||||
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
|
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
|
||||||
tags: [jellyfin, dankserver, media-server]
|
tags: [jellyfin, dankserver, media-server]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
|
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
|
||||||
tags:
|
tags:
|
||||||
- dankserver
|
- dankserver
|
||||||
- romm
|
- romm
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: MariaDB (RomM Backend)
|
title: MariaDB (RomM Backend)
|
||||||
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
|
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
|
||||||
tags: [mariadb, romm, dankserver]
|
tags: [mariadb, romm, dankserver]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Redis (RomM Cache / Queue)
|
title: Redis (RomM Cache / Queue)
|
||||||
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
|
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
|
||||||
tags: [redis, romm, dankserver]
|
tags: [redis, romm, dankserver]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Noctalia and Niri
|
title: Noctalia and Niri
|
||||||
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
|
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
|
||||||
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
|
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: "*arr Stack"
|
title: "*arr Stack"
|
||||||
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
|
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
|
||||||
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
|
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
|
||||||
timestamp: 2026-09-04T00:00:00Z
|
timestamp: 2026-09-04T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
|
|||||||
|
|
||||||
## *arr Stack
|
## *arr Stack
|
||||||
|
|
||||||
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
|
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
|
||||||
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
- [[sonarr]] — TV show management
|
||||||
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
- [[radarr]] — movie management
|
||||||
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
- [[lidarr]] — music management
|
||||||
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
|
- [[readarr]] — ebook/audiobook management
|
||||||
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
|
- [[prowlarr]] — indexer management
|
||||||
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
|
- [[qbittorrent]] — torrent download client
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Prowlarr
|
title: Prowlarr
|
||||||
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
|
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
|
||||||
tags: [prowlarr, arr-stack, frankbook-pro]
|
tags: [prowlarr, arr-stack, frankbook-pro]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: qBittorrent
|
title: qBittorrent
|
||||||
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
|
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
|
||||||
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
|
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,19 +1,18 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Readarr
|
title: Readarr
|
||||||
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
|
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
|
||||||
tags: [readarr, arr-stack, frankbook-pro]
|
tags: [readarr, arr-stack, frankbook-pro]
|
||||||
timestamp: 2026-09-03T00:00:00Z
|
timestamp: 2026-09-03T00:00:00Z
|
||||||
---
|
---
|
||||||
|
|
||||||
# Readarr
|
# Readarr
|
||||||
|
|
||||||
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
|
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
|
||||||
> Readarr: its metadata became unusable and the community migration to Open
|
> metadata became unusable and the community migration to Open Library stalled.
|
||||||
> Library stalled. There is no upstream development — the GitHub repository is
|
> There is no upstream development. Alternatives: the Bookshelf community fork,
|
||||||
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
|
> or the Calibre content server. This instance still runs on a pinned image
|
||||||
> community fork, or the Calibre content server. This instance still runs on a
|
> tag; see below.
|
||||||
> pinned image tag; see below.
|
|
||||||
|
|
||||||
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
|
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
|
||||||
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
|
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
|
||||||
@@ -70,9 +69,9 @@ Restart=always
|
|||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
|
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
|
||||||
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
|
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
|
||||||
because `:latest` does not exist for this image on LSIO.
|
`:latest` does not exist for this image on LSIO.
|
||||||
- Calibre integration is folder-based, not API-based. Readarr drops files into
|
- Calibre integration is folder-based, not API-based. Readarr drops files into
|
||||||
the books root which [[Calibre-Web]] serves.
|
the books root which [[Calibre-Web]] serves.
|
||||||
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
|
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
|
||||||
@@ -82,10 +81,3 @@ Restart=always
|
|||||||
|
|
||||||
- [[ARR-Stack]] — full *arr stack documentation
|
- [[ARR-Stack]] — full *arr stack documentation
|
||||||
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
|
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
|
||||||
|
|
||||||
## Citations
|
|
||||||
|
|
||||||
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
|
|
||||||
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
|
|
||||||
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
|
|
||||||
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
|
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
|
||||||
tags:
|
tags:
|
||||||
- infrastructure
|
- infrastructure
|
||||||
timestamp: 2026-09-17
|
timestamp: 2026-09-17
|
||||||
|
|||||||
@@ -1,230 +0,0 @@
|
|||||||
---
|
|
||||||
type: reference
|
|
||||||
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
|
|
||||||
tags:
|
|
||||||
- infrastructure
|
|
||||||
- authentik
|
|
||||||
timestamp: 2026-10-19
|
|
||||||
---
|
|
||||||
|
|
||||||
# Authentik Outpost Reference
|
|
||||||
|
|
||||||
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
|
|
||||||
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
|
|
||||||
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
|
|
||||||
self-serve user-settings flow, and the live database shape. What the service is,
|
|
||||||
how it is deployed, wired and operated is [[Authentik]]; the three core
|
|
||||||
containers are [[Authentik Processes]].
|
|
||||||
|
|
||||||
## Live proxy-provider inventory
|
|
||||||
|
|
||||||
Live provider inventory (read from the `authentik-postgres` container with
|
|
||||||
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
|
||||||
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
|
||||||
members in `authentik_outposts_outpost_providers`), all with
|
|
||||||
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
|
||||||
UUIDs** (`default-authentication-flow`,
|
|
||||||
`default-provider-authorization-implicit-consent`,
|
|
||||||
`default-provider-invalidation-flow`), and each backed by an Application
|
|
||||||
(the `slug` column below):
|
|
||||||
|
|
||||||
| Provider pk | name | external_host | app slug |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
|
||||||
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
|
||||||
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
|
||||||
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
|
||||||
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
|
||||||
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
|
||||||
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
|
||||||
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
|
||||||
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
|
||||||
|
|
||||||
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
|
||||||
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
|
|
||||||
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
|
||||||
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
|
||||||
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
|
||||||
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
|
||||||
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
|
||||||
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
|
||||||
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
|
||||||
below rejects), and pk 14 `Open WebUI`
|
|
||||||
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
|
||||||
`external_host` (the runbook-recipe style) and bare origins both intercept
|
|
||||||
correctly — the outpost matches on hostname.
|
|
||||||
|
|
||||||
## Provisioning rules for proxy providers
|
|
||||||
|
|
||||||
Two provisioning rules, each verified live by an outage:
|
|
||||||
|
|
||||||
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
|
||||||
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
|
||||||
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
|
||||||
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
|
||||||
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
|
||||||
forever, and **everything behind the outpost dies at the next outpost
|
|
||||||
reload** — including hosts whose providers were fine (verified 2026-09-02
|
|
||||||
when three providers provisioned without any flows took down every guarded
|
|
||||||
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
|
||||||
`default-authentication-flow`,
|
|
||||||
`default-provider-authorization-implicit-consent` (the implicit-consent
|
|
||||||
authorization flow, `default-invalidation-flow`) and the retry loop
|
|
||||||
recovers on its next 3-second cycle. The silent killer: while the outpost
|
|
||||||
process keeps its last good in-memory config, everything works — the
|
|
||||||
outage only surfaces on restart/reload.
|
|
||||||
2. **Every proxy provider needs an attached Application.** The outpost
|
|
||||||
resolves the post-login redirect through the provider's application; a
|
|
||||||
provider with no app completes the login (authentik session established)
|
|
||||||
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
|
||||||
the app. No error is logged anywhere — the SSO "works" and the app never
|
|
||||||
appears. Create the Application with `provider_id` set (the UI does this
|
|
||||||
automatically; raw API/ORM drives must do it explicitly).
|
|
||||||
|
|
||||||
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
|
||||||
|
|
||||||
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
|
||||||
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
|
||||||
implementation diverges from upstream in ways that bit us; all verified live:
|
|
||||||
|
|
||||||
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
|
||||||
create the single-table-inheritance `Application` row; without it the flow
|
|
||||||
executor can never complete an authorize request. Create both rows.
|
|
||||||
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
|
||||||
contains `"authorization_code"` (enum value with underscore), the token
|
|
||||||
endpoint rejects every exchange with `invalid_request`.
|
|
||||||
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
|
||||||
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
|
|
||||||
POST-body client secrets are not accepted — authenticate the confidential
|
|
||||||
client with `Authorization: Basic base64(client_id:client_secret)`.
|
|
||||||
- **Provider resolution is by `client_id` query param** on the global
|
|
||||||
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
|
||||||
this build. Token/userinfo live at `/application/o/token/` and
|
|
||||||
`/application/o/userinfo/?api=1`.
|
|
||||||
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
|
||||||
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
|
||||||
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
|
||||||
becomes the *expression source* instead of the context; every call raises a
|
|
||||||
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
|
||||||
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
|
||||||
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
|
|
||||||
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
|
||||||
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
|
||||||
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
|
||||||
build until the evaluator is patched or upgraded. Clients consuming this
|
|
||||||
provider must key identity off `sub`.
|
|
||||||
- Scope mappings for this provider exist — `grafana-email-scope` and
|
|
||||||
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
|
||||||
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
|
||||||
the evaluator bug is fixed; no config change is needed then.
|
|
||||||
|
|
||||||
## LDAP provider objects (provisioned via `ak shell` ORM)
|
|
||||||
|
|
||||||
The LDAP outpost's own deployment — unit, image digest, published ports, token
|
|
||||||
env var — is in [[Authentik]]. The objects it serves:
|
|
||||||
|
|
||||||
Read from the live database (read-only `SELECT`s against the
|
|
||||||
`authentik-postgres` container, 2026-10-18):
|
|
||||||
|
|
||||||
| Object | Value |
|
|
||||||
|---|---|
|
|
||||||
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
|
||||||
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
|
||||||
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
|
||||||
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
|
|
||||||
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
|
||||||
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
|
||||||
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
|
|
||||||
|
|
||||||
Hard-won rules (the first is now contested by the live database):
|
|
||||||
|
|
||||||
- **Token intent: contested, do not trust either value blindly.** An earlier
|
|
||||||
revision asserted that a bind token with `intent=app_password` is rejected by
|
|
||||||
the flow's password stage with `Invalid password` and only
|
|
||||||
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
|
|
||||||
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
|
|
||||||
wrong or the current Jellyfin bind path is not that token. Nothing observable
|
|
||||||
settles it today: the LDAP outpost's retained log holds only its 5-minute
|
|
||||||
`initialized direct binder` / `Update providers` config cycles and no bind
|
|
||||||
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
|
||||||
requires the bind secret)*. Re-verify with a real bind before changing the
|
|
||||||
token's intent either way.
|
|
||||||
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
|
||||||
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
|
||||||
matches zero entries and the plugin logs `Found no users matching X`.
|
|
||||||
The permission is object-level on the provider; grant it via an RBAC role
|
|
||||||
and retest.
|
|
||||||
- `ak shell` import path for the provider model in 2026.8:
|
|
||||||
`from authentik.providers.ldap.models import LDAPProvider`.
|
|
||||||
*(unverified 2026-10-18: importing it outside a configured Django shell fails
|
|
||||||
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
|
||||||
auth service for this check)*
|
|
||||||
|
|
||||||
## Self-serve user-settings flow (password change)
|
|
||||||
|
|
||||||
Users change their own account details through the `default-user-settings-flow`
|
|
||||||
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
|
||||||
target of the homepage footer's "account" link. It is a `stage_configuration`
|
|
||||||
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
|
||||||
user.
|
|
||||||
|
|
||||||
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
|
||||||
insertion order):
|
|
||||||
|
|
||||||
| order | stage | type | what it does |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
|
||||||
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
|
||||||
|
|
||||||
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
|
||||||
writes whatever the preceding `PromptStage` collected: for the key `password`
|
|
||||||
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
|
||||||
is a `password` field on the `default-user-settings` prompt
|
|
||||||
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
|
||||||
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
|
||||||
`validate_password` runs `authenticate()` against the backends to check the
|
|
||||||
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
|
||||||
2026-09-25 and removed the same day for exactly this reason.)
|
|
||||||
|
|
||||||
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
|
||||||
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
|
||||||
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
|
|
||||||
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
|
||||||
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
|
||||||
|
|
||||||
**A blank password must not set an empty password.** `set_password("")` would
|
|
||||||
make the account log in with no password, so a validation policy
|
|
||||||
(`default-user-settings-no-empty-password`, bound to the prompt via
|
|
||||||
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
|
||||||
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
|
||||||
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
|
||||||
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
|
||||||
invalid).
|
|
||||||
|
|
||||||
## Live database shape
|
|
||||||
|
|
||||||
For live DB truth prefer the postgres container, and it works without any
|
|
||||||
credential — the local socket is trusted, so a read-only audit is just
|
|
||||||
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
|
||||||
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
|
||||||
before guessing: 2026.x renamed both tables and columns, and the models are
|
|
||||||
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
|
||||||
three `*_flow_id` columns; the proxy table is
|
|
||||||
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
|
||||||
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
|
||||||
table in 2026.8), the LDAP table is
|
|
||||||
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
|
|
||||||
`authentik_core_application` attaches to a provider through `provider_id`
|
|
||||||
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
|
||||||
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
|
||||||
is the same story from the ORM side: `type` lives in the subclass tables.
|
|
||||||
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
|
||||||
|
|
||||||
## Cross-references
|
|
||||||
|
|
||||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
|
||||||
failure modes
|
|
||||||
- [[Authentik Processes]] — the postgres, server and worker containers
|
|
||||||
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
|
|
||||||
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
|
|
||||||
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
---
|
|
||||||
type: reference
|
|
||||||
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
|
|
||||||
tags:
|
|
||||||
- infrastructure
|
|
||||||
- authentik
|
|
||||||
- postgres
|
|
||||||
timestamp: 2026-10-19
|
|
||||||
---
|
|
||||||
|
|
||||||
# Authentik Processes
|
|
||||||
|
|
||||||
The three core containers of the [[Authentik]] stack on Dankclaw. The two
|
|
||||||
outpost units are not covered here — they live on [[Authentik]] and
|
|
||||||
[[Authentik Outpost Reference]]. Server and worker run the same image; the
|
|
||||||
`command` selects the role (`server` / `worker`). Data persists under
|
|
||||||
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
|
|
||||||
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
|
|
||||||
standardization; the retired stubs this page replaces recorded the pre-move
|
|
||||||
paths `/home/dank/authentik/data` (server and worker) and
|
|
||||||
`/home/dank/authentik/postgres` (database).
|
|
||||||
|
|
||||||
## authentik-postgres
|
|
||||||
|
|
||||||
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
|
|
||||||
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
|
|
||||||
PostgreSQL in 2025.10 (Redis removed).
|
|
||||||
|
|
||||||
- **Image:** `docker.io/library/postgres:16-alpine`
|
|
||||||
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
|
|
||||||
volumes)
|
|
||||||
- **Health:** `pg_isready -d authentik -U authentik`
|
|
||||||
- **Retired wording:** the stub called this a "Compose service … in
|
|
||||||
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
|
|
||||||
- The side-by-side with `gitea-postgres` (network, published ports, secret
|
|
||||||
source, boot persistence) is owned by [[Postgres]].
|
|
||||||
|
|
||||||
## authentik-server
|
|
||||||
|
|
||||||
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
|
||||||
It also serves the login page at `https://auth.franksdank.cloud`.
|
|
||||||
|
|
||||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
|
|
||||||
(`authentik-server.service`)
|
|
||||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
|
|
||||||
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
|
|
||||||
live here
|
|
||||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
|
||||||
worker)
|
|
||||||
- **Health:** recorded by the retired stub as
|
|
||||||
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
|
|
||||||
owns the live probe.
|
|
||||||
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
|
|
||||||
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
|
|
||||||
integration.
|
|
||||||
|
|
||||||
## authentik-worker
|
|
||||||
|
|
||||||
The Authentik worker — background tasks: event processing, provider
|
|
||||||
synchronization, scheduled flows, and certificate renewal.
|
|
||||||
|
|
||||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
|
|
||||||
(`authentik-worker.service`)
|
|
||||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
|
|
||||||
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
|
|
||||||
the data dir)
|
|
||||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
|
||||||
server)
|
|
||||||
- **No port** — the worker has no HTTP endpoint.
|
|
||||||
|
|
||||||
## Cross-references
|
|
||||||
|
|
||||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
|
||||||
failure modes
|
|
||||||
- [[Authentik Outpost Reference]] — provider, flow and token configuration
|
|
||||||
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
|
|
||||||
@@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
|
|||||||
they handle their own session. Grafana is the exception that is both: it is a
|
they handle their own session. Grafana is the exception that is both: it is a
|
||||||
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
|
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
|
||||||
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
||||||
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
|
`dankclaw/Caddyfile:374-377`; live provider table below).
|
||||||
|
|
||||||
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
|
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
|
||||||
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
|
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
|
||||||
@@ -35,13 +35,6 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
|||||||
pre-quadlet era, retired.
|
pre-quadlet era, retired.
|
||||||
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
|
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
|
||||||
|
|
||||||
**Split off from this page:** [[Authentik Processes]] owns the per-container
|
|
||||||
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
|
|
||||||
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
|
|
||||||
the live proxy-provider inventory, the proxy-provider provisioning rules, the
|
|
||||||
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
|
|
||||||
design, and the live database shape.
|
|
||||||
|
|
||||||
## Why Authentik
|
## Why Authentik
|
||||||
|
|
||||||
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
|
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
|
||||||
@@ -107,10 +100,23 @@ contract:
|
|||||||
The chain terminates there — nothing dials back into Authentik from
|
The chain terminates there — nothing dials back into Authentik from
|
||||||
`:8017`.
|
`:8017`.
|
||||||
|
|
||||||
## The stack's processes
|
## The three compose services
|
||||||
|
|
||||||
Per-container facts — unit path, image and role, port, run-as, data directory,
|
| Service | Role | Image |
|
||||||
health probe — are owned by [[Authentik Processes]].
|
|---|---|---|
|
||||||
|
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
|
||||||
|
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||||
|
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||||
|
|
||||||
|
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
|
||||||
|
moved to PostgreSQL in 2025.10 (Redis removed).
|
||||||
|
- **Server** and **worker** run the same image; the `command` selects the role
|
||||||
|
(`server` / `worker`).
|
||||||
|
- The worker runs as `0:0` (rootless: container root maps to the host user that
|
||||||
|
owns the data dir).
|
||||||
|
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
|
||||||
|
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
|
||||||
|
in the 2026-09-01 quadlet standardization).
|
||||||
|
|
||||||
## Caddy integration
|
## Caddy integration
|
||||||
|
|
||||||
@@ -136,9 +142,104 @@ routing table, and the Authentik proxy-provider list must be the same set of
|
|||||||
hosts. A guarded host with no provider 404s at login time; a provider with no
|
hosts. A guarded host with no provider 404s at login time; a provider with no
|
||||||
DNS record breaks off-LAN clients.
|
DNS record breaks off-LAN clients.
|
||||||
|
|
||||||
The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
|
Live provider inventory (read from the `authentik-postgres` container with
|
||||||
the two provisioning rules the outpost enforces, and this fork's OAuth2
|
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
||||||
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
|
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
||||||
|
members in `authentik_outposts_outpost_providers`), all with
|
||||||
|
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
||||||
|
UUIDs** (`default-authentication-flow`,
|
||||||
|
`default-provider-authorization-implicit-consent`,
|
||||||
|
`default-provider-invalidation-flow`), and each backed by an Application
|
||||||
|
(the `slug` column below):
|
||||||
|
|
||||||
|
| Provider pk | name | external_host | app slug |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
||||||
|
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
||||||
|
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
||||||
|
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
||||||
|
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
||||||
|
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
||||||
|
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
||||||
|
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
||||||
|
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
||||||
|
|
||||||
|
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
||||||
|
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
|
||||||
|
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
||||||
|
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
||||||
|
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
||||||
|
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
||||||
|
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
||||||
|
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
||||||
|
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
||||||
|
below rejects), and pk 14 `Open WebUI`
|
||||||
|
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
||||||
|
`external_host` (the runbook-recipe style) and bare origins both intercept
|
||||||
|
correctly — the outpost matches on hostname.
|
||||||
|
|
||||||
|
Two provisioning rules, each verified live by an outage:
|
||||||
|
|
||||||
|
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
||||||
|
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
||||||
|
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
||||||
|
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
||||||
|
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
||||||
|
forever, and **everything behind the outpost dies at the next outpost
|
||||||
|
reload** — including hosts whose providers were fine (verified 2026-09-02
|
||||||
|
when three providers provisioned without any flows took down every guarded
|
||||||
|
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
||||||
|
`default-authentication-flow`,
|
||||||
|
`default-provider-authorization-implicit-consent` (the implicit-consent
|
||||||
|
authorization flow, `default-invalidation-flow`) and the retry loop
|
||||||
|
recovers on its next 3-second cycle. The silent killer: while the outpost
|
||||||
|
process keeps its last good in-memory config, everything works — the
|
||||||
|
outage only surfaces on restart/reload.
|
||||||
|
2. **Every proxy provider needs an attached Application.** The outpost
|
||||||
|
resolves the post-login redirect through the provider's application; a
|
||||||
|
provider with no app completes the login (authentik session established)
|
||||||
|
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
||||||
|
the app. No error is logged anywhere — the SSO "works" and the app never
|
||||||
|
appears. Create the Application with `provider_id` set (the UI does this
|
||||||
|
automatically; raw API/ORM drives must do it explicitly).
|
||||||
|
|
||||||
|
|
||||||
|
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
||||||
|
|
||||||
|
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
||||||
|
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
||||||
|
implementation diverges from upstream in ways that bit us; all verified live:
|
||||||
|
|
||||||
|
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
||||||
|
create the single-table-inheritance `Application` row; without it the flow
|
||||||
|
executor can never complete an authorize request. Create both rows.
|
||||||
|
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
||||||
|
contains `"authorization_code"` (enum value with underscore), the token
|
||||||
|
endpoint rejects every exchange with `invalid_request`.
|
||||||
|
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
||||||
|
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
|
||||||
|
POST-body client secrets are not accepted — authenticate the confidential
|
||||||
|
client with `Authorization: Basic base64(client_id:client_secret)`.
|
||||||
|
- **Provider resolution is by `client_id` query param** on the global
|
||||||
|
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
||||||
|
this build. Token/userinfo live at `/application/o/token/` and
|
||||||
|
`/application/o/userinfo/?api=1`.
|
||||||
|
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
||||||
|
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
||||||
|
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
||||||
|
becomes the *expression source* instead of the context; every call raises a
|
||||||
|
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
||||||
|
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
||||||
|
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
|
||||||
|
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
||||||
|
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
||||||
|
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
||||||
|
build until the evaluator is patched or upgraded. Clients consuming this
|
||||||
|
provider must key identity off `sub`.
|
||||||
|
- Scope mappings for this provider exist — `grafana-email-scope` and
|
||||||
|
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
||||||
|
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
||||||
|
the evaluator bug is fixed; no config change is needed then.
|
||||||
|
|
||||||
## LDAP outpost (Jellyfin auth, live 2026-09-01)
|
## LDAP outpost (Jellyfin auth, live 2026-09-01)
|
||||||
|
|
||||||
@@ -166,9 +267,44 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
|
|||||||
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
|
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
|
||||||
from the host `dankclaw/.env` (git-ignored).
|
from the host `dankclaw/.env` (git-ignored).
|
||||||
|
|
||||||
The objects behind it — LDAP provider, bind DN, service account and bind token,
|
### Authentik-side objects (provisioned via `ak shell` ORM)
|
||||||
RBAC grant, groups, application slug — and the hard-won bind rules are in
|
|
||||||
[[Authentik Outpost Reference]].
|
Read from the live database (read-only `SELECT`s against the
|
||||||
|
`authentik-postgres` container, 2026-10-18):
|
||||||
|
|
||||||
|
| Object | Value |
|
||||||
|
|---|---|
|
||||||
|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
||||||
|
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
||||||
|
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
||||||
|
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
|
||||||
|
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
||||||
|
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
||||||
|
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
|
||||||
|
|
||||||
|
Hard-won rules (the first is now contested by the live database):
|
||||||
|
|
||||||
|
- **Token intent: contested, do not trust either value blindly.** This page
|
||||||
|
long asserted that a bind token with `intent=app_password` is rejected by the
|
||||||
|
flow's password stage with `Invalid password` and only `intent=verification`
|
||||||
|
passes. The live token for `svc-jellyfin-ldap` is `app_password`
|
||||||
|
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
|
||||||
|
current Jellyfin bind path is not that token. Nothing observable settles it
|
||||||
|
today: the LDAP outpost's retained log holds only its 5-minute
|
||||||
|
`initialized direct binder` / `Update providers` config cycles and no bind
|
||||||
|
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
||||||
|
requires the bind secret)*. Re-verify with a real bind before changing the
|
||||||
|
token's intent either way.
|
||||||
|
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
||||||
|
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
||||||
|
matches zero entries and the plugin logs `Found no users matching X`.
|
||||||
|
The permission is object-level on the provider; grant it via an RBAC role
|
||||||
|
and retest.
|
||||||
|
- `ak shell` import path for the provider model in 2026.8:
|
||||||
|
`from authentik.providers.ldap.models import LDAPProvider`.
|
||||||
|
*(unverified 2026-10-18: importing it outside a configured Django shell fails
|
||||||
|
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
||||||
|
auth service for this check)*
|
||||||
|
|
||||||
## Configuration (secrets)
|
## Configuration (secrets)
|
||||||
|
|
||||||
@@ -204,8 +340,7 @@ every outpost session (see the ORM playbook).
|
|||||||
Applications** → add an application + a **Proxy** provider whose
|
Applications** → add an application + a **Proxy** provider whose
|
||||||
`external_host` is exactly the browser-facing origin
|
`external_host` is exactly the browser-facing origin
|
||||||
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
|
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
|
||||||
flows set (see the provisioning rules in
|
flows set (see the provisioning pitfall above). Attach the provider to the
|
||||||
[[Authentik Outpost Reference]]). Attach the provider to the
|
|
||||||
embedded outpost.
|
embedded outpost.
|
||||||
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
|
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
|
||||||
the app with the correct redirect URI, and paste the `client_id` /
|
the app with the correct redirect URI, and paste the `client_id` /
|
||||||
@@ -238,6 +373,47 @@ every outpost session (see the ORM playbook).
|
|||||||
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
|
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
|
||||||
Authentik migrates the schema on startup.
|
Authentik migrates the schema on startup.
|
||||||
|
|
||||||
|
## Self-serve user-settings flow (password change)
|
||||||
|
|
||||||
|
Users change their own account details through the `default-user-settings-flow`
|
||||||
|
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
||||||
|
target of the homepage footer's "account" link. It is a `stage_configuration`
|
||||||
|
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
||||||
|
user.
|
||||||
|
|
||||||
|
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
||||||
|
insertion order):
|
||||||
|
|
||||||
|
| order | stage | type | what it does |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
||||||
|
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
||||||
|
|
||||||
|
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
||||||
|
writes whatever the preceding `PromptStage` collected: for the key `password`
|
||||||
|
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
||||||
|
is a `password` field on the `default-user-settings` prompt
|
||||||
|
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
||||||
|
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
||||||
|
`validate_password` runs `authenticate()` against the backends to check the
|
||||||
|
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
||||||
|
2026-09-25 and removed the same day for exactly this reason.)
|
||||||
|
|
||||||
|
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
||||||
|
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
||||||
|
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
|
||||||
|
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
||||||
|
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
||||||
|
|
||||||
|
**A blank password must not set an empty password.** `set_password("")` would
|
||||||
|
make the account log in with no password, so a validation policy
|
||||||
|
(`default-user-settings-no-empty-password`, bound to the prompt via
|
||||||
|
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
||||||
|
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
||||||
|
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
||||||
|
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
||||||
|
invalid).
|
||||||
|
|
||||||
## Outpost claim cache (`proxysession`) — 403 after a group add
|
## Outpost claim cache (`proxysession`) — 403 after a group add
|
||||||
|
|
||||||
The embedded outpost does not re-evaluate group membership per request. Each
|
The embedded outpost does not re-evaluate group membership per request. Each
|
||||||
@@ -281,7 +457,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
|
|||||||
(the inventory table carries both bare origins and trailing slashes;
|
(the inventory table carries both bare origins and trailing slashes;
|
||||||
routing matches on hostname, verified 2026-09-02).
|
routing matches on hostname, verified 2026-09-02).
|
||||||
`internal_host = http://caddy:8017`, all three flows set, and an
|
`internal_host = http://caddy:8017`, all three flows set, and an
|
||||||
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
|
Application attached — the two provisioning rules above.
|
||||||
2. **Route flip in ONE commit/deploy**: the vhost becomes
|
2. **Route flip in ONE commit/deploy**: the vhost becomes
|
||||||
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
|
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
|
||||||
inside the `http://:8017` listener add `@<svc> host <vhost>` +
|
inside the `http://:8017` listener add `@<svc> host <vhost>` +
|
||||||
@@ -345,9 +521,22 @@ EOF
|
|||||||
is no `authentik.policies.group` module in this build.
|
is no `authentik.policies.group` module in this build.
|
||||||
*(unverified 2026-10-18: these module paths were not re-executed — importing
|
*(unverified 2026-10-18: these module paths were not re-executed — importing
|
||||||
outside a configured Django shell fails with `ImproperlyConfigured`)*
|
outside a configured Django shell fails with `ImproperlyConfigured`)*
|
||||||
- The live database's table and column names (the 2026.x renames, the MTI parent
|
- For live DB truth prefer the postgres container, and it works without any
|
||||||
keys, the read-only `psql` audit recipe) are reference lookup:
|
credential — the local socket is trusted, so a read-only audit is just
|
||||||
[[Authentik Outpost Reference]].
|
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
||||||
|
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
||||||
|
before guessing: 2026.x renamed both tables and columns, and the models are
|
||||||
|
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
||||||
|
three `*_flow_id` columns; the proxy table is
|
||||||
|
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
||||||
|
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
||||||
|
table in 2026.8), the LDAP table is
|
||||||
|
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
|
||||||
|
`authentik_core_application` attaches to a provider through `provider_id`
|
||||||
|
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
||||||
|
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
||||||
|
is the same story from the ORM side: `type` lives in the subclass tables.
|
||||||
|
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
||||||
|
|
||||||
## User provisioning playbook (user-level account)
|
## User provisioning playbook (user-level account)
|
||||||
|
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
|
|||||||
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
|
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
|
||||||
Basic client auth, form-encoded token body, and the fork's userinfo-claims
|
Basic client auth, form-encoded token body, and the fork's userinfo-claims
|
||||||
bug that leaves identity at the `sub` hash only) is in
|
bug that leaves identity at the `sub` hash only) is in
|
||||||
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
||||||
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
|
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
|
||||||
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
|
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
|
||||||
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
|
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Homepage Status Probes
|
title: Homepage Status Probes
|
||||||
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
|
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
|
||||||
tags: [homepage, status, probes, e2e, homepage-status]
|
tags: [homepage, status, probes, e2e, homepage-status]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
|
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
|
||||||
tags:
|
tags:
|
||||||
- infrastructure
|
- infrastructure
|
||||||
- homepage
|
- homepage
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Observability Provisioning
|
title: Observability Provisioning
|
||||||
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
|
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
|
||||||
tags: [observability, prometheus, grafana, provisioning, deployment]
|
tags: [observability, prometheus, grafana, provisioning, deployment]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: "Observability on Dankclaw"
|
title: "Observability on Dankclaw"
|
||||||
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
||||||
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
|
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
|
||||||
timestamp: 2026-09-11
|
timestamp: 2026-09-11
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
|
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
|
||||||
tags:
|
tags:
|
||||||
- infrastructure
|
- infrastructure
|
||||||
- prometheus
|
- prometheus
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
|
||||||
|
tags:
|
||||||
|
- infrastructure
|
||||||
|
- authentik
|
||||||
|
- postgres
|
||||||
|
timestamp: 2026-08-28
|
||||||
|
---
|
||||||
|
|
||||||
|
# authentik-postgres
|
||||||
|
|
||||||
|
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
|
||||||
|
OIDC provider. It is the only stateful store in the stack (users, flows,
|
||||||
|
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
|
||||||
|
now live here.
|
||||||
|
|
||||||
|
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
|
||||||
|
- **Image:** `docker.io/library/postgres:16-alpine`
|
||||||
|
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
|
||||||
|
- **Health:** `pg_isready -d authentik -U authentik`
|
||||||
|
|
||||||
|
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
|
||||||
|
(`pg_dump`), and failure modes.
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
|
||||||
|
tags:
|
||||||
|
- infrastructure
|
||||||
|
- authentik
|
||||||
|
timestamp: 2026-09-16
|
||||||
|
---
|
||||||
|
|
||||||
|
# authentik-server
|
||||||
|
|
||||||
|
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
||||||
|
This is the service the Caddy `forward_auth` vhosts proxy to
|
||||||
|
(`authentik-server:9000`). It also serves the login page at
|
||||||
|
`https://auth.franksdank.cloud`.
|
||||||
|
|
||||||
|
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
|
||||||
|
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
|
||||||
|
- **Port:** `9000` (published; the outpost + admin + login all live here)
|
||||||
|
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
|
||||||
|
- **Health:** `http://localhost:9000/-/health/`
|
||||||
|
|
||||||
|
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
|
||||||
|
forward_auth pattern, first-time setup, and failure modes.
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
description: "The Authentik worker — background tasks: event processing, provider"
|
||||||
|
tags:
|
||||||
|
- infrastructure
|
||||||
|
- authentik
|
||||||
|
timestamp: 2026-09-16
|
||||||
|
---
|
||||||
|
|
||||||
|
# authentik-worker
|
||||||
|
|
||||||
|
The Authentik worker — background tasks: event processing, provider
|
||||||
|
synchronization, scheduled flows, and certificate renewal. It runs the same
|
||||||
|
image as the server; the `command: worker` selects the role.
|
||||||
|
|
||||||
|
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
|
||||||
|
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
|
||||||
|
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
|
||||||
|
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
|
||||||
|
- **No port** — the worker has no HTTP endpoint.
|
||||||
|
|
||||||
|
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
|
||||||
|
modes.
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: Dankserver System Specs
|
title: Dankserver System Specs
|
||||||
description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
|
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
|
||||||
tags:
|
tags:
|
||||||
- dankserver
|
- dankserver
|
||||||
- hardware
|
- hardware
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: research
|
||||||
title: Dual R9700 Cooperation
|
title: Dual R9700 Cooperation
|
||||||
description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
|
description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
|
||||||
tags:
|
tags:
|
||||||
- frank-medium
|
- frank-medium
|
||||||
- rocm
|
- rocm
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: research
|
||||||
title: FastFlowLM NPU Inference
|
title: FastFlowLM NPU Inference
|
||||||
description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
|
description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
|
||||||
tags:
|
tags:
|
||||||
- fastflowlm
|
- fastflowlm
|
||||||
- npu
|
- npu
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: FreeToken Builds
|
title: FreeToken Builds
|
||||||
description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
|
description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
|
||||||
tags: [freetoken, build, dankclaw, vllm, mtp]
|
tags: [freetoken, build, dankclaw, vllm, mtp]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: GGUF Compatibility and Runtime Lock-in
|
title: GGUF Compatibility and Runtime Lock-in
|
||||||
description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
|
description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
|
||||||
tags: [gguf, llama.cpp, forks, quantization, speculative]
|
tags: [gguf, llama.cpp, forks, quantization, speculative]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Heterogeneous Split Triage
|
title: Heterogeneous Split Triage
|
||||||
description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
|
description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
|
||||||
tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
|
tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: LLM Bench Windows and Measurement
|
title: LLM Bench Windows and Measurement
|
||||||
description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
|
description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
|
||||||
tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
|
tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: LLM Performance Diagnosis
|
title: LLM Performance Diagnosis
|
||||||
description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
|
description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
|
||||||
tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
|
tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Llama Cpp Multi-Device Splits
|
title: Llama Cpp Multi-Device Splits
|
||||||
description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
|
description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
|
||||||
tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
|
tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Llama Cpp Serving Sizing
|
title: Llama Cpp Serving Sizing
|
||||||
description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
|
description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
|
||||||
tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
|
tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Model Swap and Rollback
|
title: Model Swap and Rollback
|
||||||
description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
|
description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
|
||||||
tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
|
tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: PCIe Link and Atomics
|
title: PCIe Link and Atomics
|
||||||
description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
|
description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
|
||||||
tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
|
tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Ethernet Bandwidth Math
|
title: Ethernet Bandwidth Math
|
||||||
description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
|
description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
|
||||||
tags:
|
tags:
|
||||||
- pipeline-parallelism
|
- pipeline-parallelism
|
||||||
- research
|
- research
|
||||||
timestamp: 2026-10-19
|
timestamp: 2026-08-17
|
||||||
---
|
---
|
||||||
|
|
||||||
# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
|
# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
|
||||||
@@ -147,11 +147,19 @@ bytes_per_boundary = batch_size × hidden_size × 2 [fp16 = 2 bytes]
|
|||||||
- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
|
- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
|
||||||
- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
|
- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
|
||||||
|
|
||||||
**Worked examples (1 boundary = 2 PP stages):**
|
**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
|
||||||
```
|
```
|
||||||
27B, batch=1: 1 boundary × 4096 × 2 bytes = 8,192 bytes/tok = 8 KB
|
1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
|
||||||
27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288 = 512 KB/tok
|
```
|
||||||
36B, batch=1: 1 boundary × 6144 × 2 bytes = 12,288 bytes/tok = 12 KB
|
|
||||||
|
**Example — same model, batch=64:**
|
||||||
|
```
|
||||||
|
1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example — 36B model, 2 PP stages, batch=1:**
|
||||||
|
```
|
||||||
|
1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
|
||||||
```
|
```
|
||||||
|
|
||||||
### 1.2 Tensor Parallelism (TP) — for comparison
|
### 1.2 Tensor Parallelism (TP) — for comparison
|
||||||
@@ -236,13 +244,57 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2 [L all-reduces total
|
|||||||
| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
|
| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
|
||||||
| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
|
| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
|
||||||
|
|
||||||
### 2.3 Feasibility
|
### 2.3 Feasibility matrix
|
||||||
|
|
||||||
Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
|
Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
|
||||||
**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
|
|
||||||
feasible on every link class — even the heaviest, 27B at batch=64 and 100
|
**27B model — PP (1 boundary), batch=1:**
|
||||||
tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
|
|
||||||
except batch=64:
|
| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
|
||||||
|
|
||||||
|
**27B model — PP (1 boundary), batch=16:**
|
||||||
|
|
||||||
|
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
|
||||||
|
|
||||||
|
**27B model — PP (1 boundary), batch=64:**
|
||||||
|
|
||||||
|
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
|
||||||
|
|
||||||
|
**27B model — TP (32 layers), batch=1:**
|
||||||
|
|
||||||
|
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
|
||||||
|
|
||||||
|
**27B model — TP (32 layers), batch=16:**
|
||||||
|
|
||||||
|
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
|
||||||
|
|
||||||
|
**27B model — TP (32 layers), batch=64:**
|
||||||
|
|
||||||
|
| | 1 GbE | 2.5 GbE | 10 GbE |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
|
||||||
|
| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
|
||||||
|
| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
|
||||||
|
|
||||||
**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
|
**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
|
||||||
|
|
||||||
@@ -316,7 +368,10 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
|
|||||||
**Petals [1]:**
|
**Petals [1]:**
|
||||||
- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
|
- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
|
||||||
- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
|
- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
|
||||||
|
- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
|
||||||
- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
|
- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
|
||||||
|
- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
|
||||||
|
- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)
|
||||||
|
|
||||||
**llama.cpp RPC [2]:**
|
**llama.cpp RPC [2]:**
|
||||||
- Model weights and KV cache distributed across nodes proportionally to memory
|
- Model weights and KV cache distributed across nodes proportionally to memory
|
||||||
@@ -438,17 +493,38 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute
|
|||||||
|
|
||||||
## Appendix: Full bandwidth formulae
|
## Appendix: Full bandwidth formulae
|
||||||
|
|
||||||
### Pipeline and Tensor Parallelism (generation)
|
### Pipeline Parallelism (generation, batch=1)
|
||||||
|
|
||||||
`N` is the batch size; `N = 1` is the batch=1 form.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
bytes_per_boundary_per_tok = hidden_size × N × 2 [fp16, PP boundary]
|
bytes_per_boundary_per_tok = hidden_size × 2 [fp16]
|
||||||
bytes_per_layer_allreduce = hidden_size × N × 2 [TP, one all-reduce per layer]
|
total_bytes_per_tok = num_boundaries × hidden_size × 2
|
||||||
total_bytes_per_tok = num_boundaries × hidden_size × N × 2 [PP]
|
|
||||||
total_bytes_per_tok = num_layers × hidden_size × N × 2 [TP]
|
|
||||||
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
|
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
|
||||||
|
```
|
||||||
|
|
||||||
# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
|
### Pipeline Parallelism (generation, batch=N)
|
||||||
|
|
||||||
|
```
|
||||||
|
bytes_per_boundary_per_tok = hidden_size × N × 2
|
||||||
|
total_bytes_per_tok = num_boundaries × hidden_size × N × 2
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tensor Parallelism (generation, batch=1)
|
||||||
|
|
||||||
|
```
|
||||||
|
bytes_per_layer_allreduce = hidden_size × 2
|
||||||
|
total_bytes_per_tok = num_layers × hidden_size × 2
|
||||||
|
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tensor Parallelism (generation, batch=N)
|
||||||
|
|
||||||
|
```
|
||||||
|
total_bytes_per_tok = num_layers × hidden_size × N × 2
|
||||||
|
```
|
||||||
|
|
||||||
|
### With 8-bit quantization (like Petals)
|
||||||
|
|
||||||
|
```
|
||||||
|
# Simply divide all fp16 byte counts by 2
|
||||||
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
|
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
|
||||||
```
|
```
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Expected Performance
|
title: Expected Performance
|
||||||
description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
|
description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
|
||||||
tags:
|
tags:
|
||||||
- pipeline-parallelism
|
- pipeline-parallelism
|
||||||
- performance
|
- performance
|
||||||
@@ -13,7 +13,7 @@ timestamp: 2026-08-14
|
|||||||
|
|
||||||
This page estimates the speed of a 25-40B model split across two machines
|
This page estimates the speed of a 25-40B model split across two machines
|
||||||
over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
|
over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
|
||||||
and the mixed-vendor measurements in [[x4-pair-experiment]].
|
and the measured numbers in [[Mixed-Vendor Experiments]].
|
||||||
|
|
||||||
## The Bandwidth Ceiling Is Not the Limit
|
## The Bandwidth Ceiling Is Not the Limit
|
||||||
|
|
||||||
@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
|
|||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [[Ethernet Bandwidth Math]]
|
- [[Ethernet Bandwidth Math]]
|
||||||
- [[Qwen3.8 PP Deployment]]
|
- [[Mixed-Vendor Experiments]]
|
||||||
- [[Sample Script Set]]
|
- [[Sample Script Set]]
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: "Mixed-Vendor Experiments"
|
||||||
|
description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
|
||||||
|
tags:
|
||||||
|
- pipeline-parallelism
|
||||||
|
- research
|
||||||
|
timestamp: 2026-08-14
|
||||||
|
---
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: "Qwen3.6 27B Spec Methods"
|
||||||
|
description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
|
||||||
|
tags:
|
||||||
|
- pipeline-parallelism
|
||||||
|
- research
|
||||||
|
timestamp: 2026-08-14
|
||||||
|
---
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
|
description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- pipeline
|
- pipeline
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Sample Script Set
|
title: Sample Script Set
|
||||||
description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
|
description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
|
||||||
tags:
|
tags:
|
||||||
- pipeline-parallelism
|
- pipeline-parallelism
|
||||||
- sample
|
- sample
|
||||||
@@ -255,7 +255,7 @@ is 30 GiB.
|
|||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [[Qwen3.8 PP Deployment]]
|
- [[llama.cpp RPC]]
|
||||||
- [[Expected Performance]]
|
- [[Expected Performance]]
|
||||||
- [[Ethernet Bandwidth Math]]
|
- [[Ethernet Bandwidth Math]]
|
||||||
- [[DFlash DSpark Speculative Decoding]]
|
- [[Qwen3.6 27B Spec Methods]]
|
||||||
@@ -28,24 +28,28 @@ graph LR
|
|||||||
end
|
end
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- [[llama.cpp RPC]] — the working, production-capable path for two
|
||||||
heterogeneous nodes over ethernet.
|
heterogeneous nodes over ethernet.
|
||||||
|
- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
|
||||||
multiprocessing and does not compose with speculative decoding yet.
|
multiprocessing and does not compose with speculative decoding yet.
|
||||||
|
|
||||||
## What the Research Found
|
## What the Research Found
|
||||||
|
|
||||||
- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
|
- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
|
||||||
(about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
|
(about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
|
||||||
tokens per second on paper; the real limit is latency and RPC overhead.
|
tokens per second on paper; the real limit is latency and RPC overhead.
|
||||||
|
- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
|
||||||
works today via llama.cpp RPC with separate CUDA and ROCm builds. The
|
works today via llama.cpp RPC with separate CUDA and ROCm builds. The
|
||||||
exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
|
exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
|
||||||
published numbers yet.
|
published numbers yet.
|
||||||
|
- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
|
||||||
methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
|
methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
|
||||||
- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
|
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
|
||||||
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
|
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
|
- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
|
||||||
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
|
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
|
||||||
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
|
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
|
||||||
`--rpc`, default split is proportional to reported free memory (pin 1,1),
|
`--rpc`, default split is proportional to reported free memory (pin 1,1),
|
||||||
@@ -53,7 +57,7 @@ graph LR
|
|||||||
|
|
||||||
## Sample Scripts
|
## Sample Scripts
|
||||||
|
|
||||||
- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
|
- [[Sample Script Set]] — a two-node llama.cpp RPC split of
|
||||||
Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
|
Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
|
||||||
the largest context that fits. For educational reading only. Do not run
|
the largest context that fits. For educational reading only. Do not run
|
||||||
it in production.
|
it in production.
|
||||||
@@ -64,8 +68,3 @@ graph LR
|
|||||||
- [[frank-zealot]]
|
- [[frank-zealot]]
|
||||||
- [[Dankserver System Specs]]
|
- [[Dankserver System Specs]]
|
||||||
- [[Dankclaw System Specs]]
|
- [[Dankclaw System Specs]]
|
||||||
|
|
||||||
## Pages
|
|
||||||
|
|
||||||
| Page | Owns |
|
|
||||||
|---|---|
|
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: "llama.cpp RPC"
|
||||||
|
description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
|
||||||
|
tags:
|
||||||
|
- pipeline-parallelism
|
||||||
|
- research
|
||||||
|
timestamp: 2026-08-14
|
||||||
|
---
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: "vLLM Pipeline Parallelism"
|
||||||
|
description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
|
||||||
|
tags:
|
||||||
|
- pipeline-parallelism
|
||||||
|
- research
|
||||||
|
timestamp: 2026-08-14
|
||||||
|
---
|
||||||
@@ -1,13 +1,3 @@
|
|||||||
---
|
|
||||||
type: "howto"
|
|
||||||
title: "Quantization Production"
|
|
||||||
description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
|
|
||||||
tags:
|
|
||||||
- llm
|
|
||||||
- quantization
|
|
||||||
timestamp: 2026-10-19
|
|
||||||
---
|
|
||||||
|
|
||||||
# Quantization Production — making our own quants, per card and runtime
|
# Quantization Production — making our own quants, per card and runtime
|
||||||
|
|
||||||
**Research page (2026-10-08).** Question: how do WE produce quantizations
|
**Research page (2026-10-08).** Question: how do WE produce quantizations
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: RDNA4 Container Serving
|
title: RDNA4 Container Serving
|
||||||
description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
|
description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
|
||||||
tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
|
tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: RDNA4 MoE Expert Offload VRAM Budget
|
title: RDNA4 MoE Expert Offload VRAM Budget
|
||||||
description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
|
description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
|
||||||
tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
|
tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
|
description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
timestamp: 2026-09-03
|
timestamp: 2026-09-03
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
|
description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
timestamp: 2026-09-03
|
timestamp: 2026-09-03
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: Tcclaviger Fork Ops
|
title: Tcclaviger Fork Ops
|
||||||
description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
|
description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
|
||||||
tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
|
tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
|
||||||
timestamp: 2026-10-05T00:00:00-07:00
|
timestamp: 2026-10-05T00:00:00-07:00
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Locator for every vLLM page in this vault."
|
description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- vllm
|
- vllm
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: research
|
||||||
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
|
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
|
||||||
description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
|
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
|
||||||
tags:
|
tags:
|
||||||
- dankclaw
|
- dankclaw
|
||||||
- iommu
|
- iommu
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: research
|
||||||
title: Frontier MoE on Dankserver — what fits, on which runtime
|
title: Frontier MoE on Dankserver — what fits, on which runtime
|
||||||
description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
|
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
|
||||||
tags:
|
tags:
|
||||||
- dankserver
|
- dankserver
|
||||||
- r9700
|
- r9700
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
|
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
|
||||||
description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
|
description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot.
|
||||||
tags:
|
tags:
|
||||||
- llamacpp
|
- llamacpp
|
||||||
- cuda
|
- cuda
|
||||||
@@ -9,7 +9,6 @@ tags:
|
|||||||
- moe
|
- moe
|
||||||
- ornith
|
- ornith
|
||||||
- mtp
|
- mtp
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-08-24
|
timestamp: 2026-08-24
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -124,4 +123,4 @@ llama-server \
|
|||||||
|
|
||||||
- [[deployment]] — deploy and update process
|
- [[deployment]] — deploy and update process
|
||||||
- [[Dankserver System Specs]] — the box
|
- [[Dankserver System Specs]] — the box
|
||||||
- [[frank-redo-2026-08-17]] — fleet topology histor
|
- [[frank-redo-2026-08-17]] — fleet topology history
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
type: howto
|
type: guide
|
||||||
title: LLM Deployment
|
title: LLM Deployment
|
||||||
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
|
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
|
||||||
tags:
|
tags:
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
|
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
|
||||||
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
|
description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- frank-archon
|
- frank-archon
|
||||||
@@ -11,7 +11,6 @@ tags:
|
|||||||
- r9700
|
- r9700
|
||||||
- tp2
|
- tp2
|
||||||
- expert-offload
|
- expert-offload
|
||||||
- endpoint
|
|
||||||
timestamp: 2026-10-18
|
timestamp: 2026-10-18
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -140,21 +139,15 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
|
|||||||
| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
|
| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
|
||||||
| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
|
| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
|
||||||
|
|
||||||
Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
|
Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
|
||||||
pages are **not** superseded here: [[frank-medium]] is the locator for that
|
(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
|
||||||
endpoint's whole shape history, including the unresolved end date of the
|
|
||||||
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
|
|
||||||
|
|
||||||
## Laws learned here
|
## Laws learned here
|
||||||
|
|
||||||
### The radiance engine
|
### The radiance engine
|
||||||
|
|
||||||
- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
|
- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
|
||||||
a **radiance** restriction, not stock vLLM's: upstream advertises `auto|fp8|fp8_e5m2|fp8_e4m3`
|
so llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
|
||||||
(`docs.vllm.ai` engine-args) and even there the advertised set is not the usable set — several
|
|
||||||
accepted values crash engine init at the attention layer's own dtype check
|
|
||||||
([vllm-project/vllm#42587](https://github.com/vllm-project/vllm/issues/42587)).
|
|
||||||
So llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
|
|
||||||
`--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
|
`--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
|
||||||
per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
|
per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
|
||||||
2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
|
2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
|
||||||
@@ -445,4 +438,3 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
|
|||||||
[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
|
[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
|
||||||
[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
|
[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
|
||||||
[[frank-medium-radiance-mxfp4]], [[LLM/index]].
|
[[frank-medium-radiance-mxfp4]], [[LLM/index]].
|
||||||
---
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
||||||
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
|
description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- frank-dragoon
|
- frank-dragoon
|
||||||
@@ -9,7 +9,6 @@ tags:
|
|||||||
- strix-halo
|
- strix-halo
|
||||||
- gfx1151
|
- gfx1151
|
||||||
- halogen
|
- halogen
|
||||||
- endpoint
|
|
||||||
timestamp: 2026-10-18
|
timestamp: 2026-10-18
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -235,4 +234,3 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
|
|||||||
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
|
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
|
||||||
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
|
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
|
||||||
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
|
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
|
||||||
---
|
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
timestamp: 2026-09-17
|
timestamp: 2026-09-17
|
||||||
type: reference
|
type: reference
|
||||||
title: frank-medium — Batch and Ubatch Sweep
|
title: frank-medium — Batch and Ubatch Sweep
|
||||||
description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
|
description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides.
|
||||||
tags:
|
tags:
|
||||||
- benchmark
|
- benchmark
|
||||||
- llamacpp
|
- llamacpp
|
||||||
@@ -11,7 +11,6 @@ tags:
|
|||||||
- dankserver
|
- dankserver
|
||||||
- layer-split
|
- layer-split
|
||||||
---
|
---
|
||||||
- retired-endpoint
|
|
||||||
|
|
||||||
# frank-medium — Batch and Ubatch Sweep
|
# frank-medium — Batch and Ubatch Sweep
|
||||||
|
|
||||||
@@ -223,5 +222,4 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
|
|||||||
difference: with no KV to attend over, per-pass overhead dominates and a
|
difference: with no KV to attend over, per-pass overhead dominates and a
|
||||||
bigger micro-batch wins; at 64k of resident KV it does not.
|
bigger micro-batch wins; at 64k of resident KV it does not.
|
||||||
- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
|
- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
|
||||||
- [[Dankserver System Specs]] — hardware inventory---
|
- [[Dankserver System Specs]] — hardware inventory
|
||||||
timestamp: 2026-09-17
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
|
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
|
||||||
description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
|
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
|
||||||
tags:
|
tags:
|
||||||
- frank-medium
|
- frank-medium
|
||||||
- dankserver
|
- dankserver
|
||||||
@@ -12,7 +12,6 @@ tags:
|
|||||||
- gfx1201
|
- gfx1201
|
||||||
- r9700
|
- r9700
|
||||||
- layer-split
|
- layer-split
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-20
|
timestamp: 2026-09-20
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -731,4 +730,3 @@ fleet.
|
|||||||
- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
|
- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
|
||||||
geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
|
geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
|
||||||
why the logical batch is free, and the harness traps
|
why the logical batch is free, and the harness traps
|
||||||
---
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: howto
|
type: runbook
|
||||||
title: frank-medium Model Swap
|
title: frank-medium Model Swap
|
||||||
description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path."
|
description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
|
||||||
tags:
|
tags:
|
||||||
- frank-medium
|
- frank-medium
|
||||||
- dankserver
|
- dankserver
|
||||||
@@ -9,7 +9,6 @@ tags:
|
|||||||
- vllm-radiance
|
- vllm-radiance
|
||||||
- huggingface
|
- huggingface
|
||||||
- runbook
|
- runbook
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-20
|
timestamp: 2026-09-20
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -156,10 +155,9 @@ back:
|
|||||||
2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
|
2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
|
||||||
3. Re-run the capability battery.
|
3. Re-run the capability battery.
|
||||||
|
|
||||||
The 2026-09-20 rollback target is the MXFP4 checkpoint with
|
The 2026-09-20 rollback target is the MXFP4 checkpoint at
|
||||||
`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that
|
`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single
|
||||||
payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a
|
safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
|
||||||
local file.
|
|
||||||
|
|
||||||
## 2026-09-20 swap record (MXFP4 → FP8)
|
## 2026-09-20 swap record (MXFP4 → FP8)
|
||||||
|
|
||||||
@@ -186,4 +184,3 @@ local file.
|
|||||||
DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
|
DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
|
||||||
smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
|
smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
|
||||||
vs 19.4 GiB).
|
vs 19.4 GiB).
|
||||||
---
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: frank-medium vllm-radiance MXFP6 Runbook
|
title: frank-medium vllm-radiance MXFP6 Runbook
|
||||||
description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
|
description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
|
||||||
tags:
|
tags:
|
||||||
- frank-medium
|
- frank-medium
|
||||||
- dankserver
|
- dankserver
|
||||||
@@ -14,7 +14,6 @@ tags:
|
|||||||
- rccl
|
- rccl
|
||||||
- pcie-atomics
|
- pcie-atomics
|
||||||
- quadlet
|
- quadlet
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-21
|
timestamp: 2026-09-21
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -38,14 +37,11 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
|
|||||||
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
|
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
|
||||||
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
|
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
|
||||||
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
|
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
|
||||||
drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
|
drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
|
||||||
below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
|
decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
|
||||||
rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
|
`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
|
||||||
`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
|
back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
|
||||||
rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
|
vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
|
||||||
`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
|
|
||||||
image back to `localhost/radiance-vllm:0.9.3` (re-pull
|
|
||||||
`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
|
|
||||||
486,464 tokens with bf16 KV and held the full 262,144 window).
|
486,464 tokens with bf16 KV and held the full 262,144 window).
|
||||||
|
|
||||||
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
|
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
|
||||||
@@ -532,8 +528,9 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
|
|||||||
with growing history) shows what happens at that limit: turns alternate between
|
with growing history) shows what happens at that limit: turns alternate between
|
||||||
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
|
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
|
||||||
and must be prefilled again, because two families per card plus their cached
|
and must be prefilled again, because two families per card plus their cached
|
||||||
prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
|
prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
|
||||||
turns on the replica that holds the prefix.
|
both replicas steady — and per-family pinning kept the fast turns on the replica
|
||||||
|
that holds the prefix.
|
||||||
|
|
||||||
Read that as a capacity budget, not a defect: two concurrent large-context
|
Read that as a capacity budget, not a defect: two concurrent large-context
|
||||||
sessions per card are comfortable, three or more of that size trade turns
|
sessions per card are comfortable, three or more of that size trade turns
|
||||||
@@ -635,4 +632,3 @@ if that person is looking. That is the honest state of "alerting" here.
|
|||||||
the same image and kernel 7.0.0-30 failing identically.
|
the same image and kernel 7.0.0-30 failing identically.
|
||||||
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
|
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
|
||||||
this page compares against.
|
this page compares against.
|
||||||
---
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: configuration
|
||||||
title: frank-medium ROCmFPX Runbook
|
title: frank-medium ROCmFPX Runbook
|
||||||
description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required."
|
description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201).
|
||||||
tags:
|
tags:
|
||||||
- frank-medium
|
- frank-medium
|
||||||
- dankserver
|
- dankserver
|
||||||
@@ -10,7 +10,6 @@ tags:
|
|||||||
- mtp
|
- mtp
|
||||||
- gfx1201
|
- gfx1201
|
||||||
- r9700
|
- r9700
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-15
|
timestamp: 2026-09-15
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,65 +0,0 @@
|
|||||||
---
|
|
||||||
type: reference
|
|
||||||
title: frank-medium (retired) — the Dankserver endpoint before frank-archon
|
|
||||||
description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree."
|
|
||||||
timestamp: 2026-10-19
|
|
||||||
tags:
|
|
||||||
- llm
|
|
||||||
- dankserver
|
|
||||||
- frank-medium
|
|
||||||
- retired
|
|
||||||
- retired-endpoint
|
|
||||||
---
|
|
||||||
|
|
||||||
# frank-medium (retired) — the Dankserver endpoint before frank-archon
|
|
||||||
|
|
||||||
`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live
|
|
||||||
endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every
|
|
||||||
shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon`
|
|
||||||
name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** —
|
|
||||||
which era page holds which payload's evidence — and carries no measurements: every
|
|
||||||
number stays in the era page that measured it.
|
|
||||||
|
|
||||||
## Eras
|
|
||||||
|
|
||||||
Windows are quoted from each era page's own banner; no date here is derived.
|
|
||||||
|
|
||||||
| Window | Runtime | Payload | Shape | Page | Rollback |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| 2026-09-15 → 2026-09-16 | `charlie12345/ROCmFPX` llama.cpp `main` at `c49ebdb`, self-contained ROCm 7.14 | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) + an MTP Q4_0 draft | 2× R9700 `-sm layer` over `ROCm0,ROCm1`, 131,072 × 1 slot, f16 KV + flash attention, draft-MTP pinned to `ROCm1`; the fork is the only loader for that tensor type | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork |
|
|
||||||
| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags |
|
|
||||||
| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape |
|
|
||||||
| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4 → FP8 E4M3 128×128 (`orcarouter/…-FP8`, same day, [[frank-medium-model-swap]]) → ParoQuant MXFP6 (`paroquant_mxfp6`) from 09-21 | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` |
|
|
||||||
|
|
||||||
`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run
|
|
||||||
on the layer-split pair at its deployed shape, and its numbers belong to that
|
|
||||||
campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of
|
|
||||||
the 2026-09-20 swap procedure — while the procedure itself is owned by
|
|
||||||
[[Model Swap and Rollback]].
|
|
||||||
|
|
||||||
## Laws
|
|
||||||
|
|
||||||
None restated. Each era page carries its own laws and links them to the technique
|
|
||||||
page that owns the mechanism ([[PCIe Link and Atomics]],
|
|
||||||
[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]],
|
|
||||||
[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]).
|
|
||||||
|
|
||||||
## Open questions
|
|
||||||
|
|
||||||
Two date claims in the era pages do not reconcile, and both are left standing rather
|
|
||||||
than silently corrected:
|
|
||||||
|
|
||||||
- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the
|
|
||||||
next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The
|
|
||||||
MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which
|
|
||||||
one directly followed the ROCmFPX fork cannot be settled from the repo.
|
|
||||||
- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while
|
|
||||||
[[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded
|
|
||||||
fortnight on that endpoint or a stale banner; nothing else in the repo covers
|
|
||||||
2026-09-26 → 10-10 for port 7770.
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
No repo files own this endpoint: its units, env and serve scripts were renamed to
|
|
||||||
`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds
|
|
||||||
retired serve scripts from these eras as reference only.
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
|
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
|
||||||
description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went."
|
description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
|
||||||
tags:
|
tags:
|
||||||
- flm
|
- flm
|
||||||
- fastflowlm
|
- fastflowlm
|
||||||
@@ -10,7 +10,6 @@ tags:
|
|||||||
- dankclaw
|
- dankclaw
|
||||||
- frank-npu
|
- frank-npu
|
||||||
- retired
|
- retired
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-11
|
timestamp: 2026-09-11
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,10 +1,9 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way."
|
description: "This page records the full redo of the inference fleet on 2026-08-17."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- frank
|
- frank
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-03
|
timestamp: 2026-09-03
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -224,4 +223,3 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17
|
|||||||
evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
|
evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
|
||||||
the main TTFT lever for pi, which re-sends the same system prompt + tool
|
the main TTFT lever for pi, which re-sends the same system prompt + tool
|
||||||
schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
|
schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
|
||||||
---
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed"
|
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed"
|
||||||
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
|
description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- frank-zealot
|
- frank-zealot
|
||||||
@@ -9,7 +9,6 @@ tags:
|
|||||||
- llama-cpp
|
- llama-cpp
|
||||||
- gguf
|
- gguf
|
||||||
- rtx3090
|
- rtx3090
|
||||||
- endpoint
|
|
||||||
timestamp: 2026-10-18
|
timestamp: 2026-10-18
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -115,7 +114,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
|
|||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | 2 × 262,144 (pool 524,288), exact f16 K+V = 16,384.00 MiB, `SPEC_TYPE=none`, batch 4096 / ubatch 2048; boot ledger KV 16,384.00 / RS 100.50 / compute 1,408.06 + 1,088.07 CUDA_Host / mmproj worst case 868.32 charged after the fit check, settled **21,176 used / 2,949 free**, `n_ctx_slot = 262144`, within 24 MiB of the projection; floors 134.1–141.5 t/s single-stream, 256.6–265.1 aggregate over two, 3,159.5 t/s prefill at 106,282, 71.5 t/s decode at 106k depth (contended — floors, not the shape's rates) | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
|
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | 2 × 262,144 (pool 524,288), exact f16 K+V = 16,384.00 MiB, `SPEC_TYPE=none`, batch 4096 / ubatch 2048; boot ledger KV 16,384.00 / RS 100.50 / compute 1,408.06 + 1,088.07 CUDA_Host / mmproj worst case 868.32 charged after the fit check, settled **21,176 used / 2,949 free**, `n_ctx_slot = 262144`, within 24 MiB of the projection; floors 134.1–141.5 t/s single-stream, 256.6–265.1 aggregate over two, 3,159.5 t/s prefill at 106,282, 71.5 t/s decode at 106k depth (contended — floors, not the shape's rates) | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
|
||||||
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | 3 × 262,144 (pool 786,432), q8_0 K+V = 13,056 MiB, settled 17,956 / 6,169 MiB; 168.7 t/s decode (three runs 168.4–168.7), ~300 t/s over three streams (102.8 each), 2,877 t/s prefill at 106,418 | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
|
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | 3 × 262,144 (pool 786,432), q8_0 K+V = 13,056 MiB, settled 17,956 / 6,169 MiB; 168.7 t/s decode (three runs 168.4–168.7), ~300 t/s over three streams (102.8 each), 2,877 t/s prefill at 106,418 | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
|
||||||
| 2026-10-18 03:14 → 13:19 (arm under the unit, then the leftover drop-in) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | arm: 2 × 200,704, batch/ubatch 3072/1536. The leftover `frank-zealot.service.d/ptq1.conf` then made the 13:16 cutover deploy boot THIS payload at 3 × 133,888 (`new slot, n_ctx = 133888` x3, journal read 2026-10-19) until the 13:19 boot | Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
|
| 2026-10-18, 24 min (arm under the unit) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | 2 × 200,704, batch/ubatch 3072/1536 | Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
|
||||||
| 2026-10-17 → 10-18 | Ternary-Bonsai-2-**27B** PQ2_0 7,206,168,928 B `3907dc16…` + Q8_0 mmproj 629,246,976 B | PrismML fork **10754** (`~/GitPackages/llama-prism/`) | 2 × 200,704 (pool 401,408), q8_0 K+V = 13,328 MiB, settled 22,664 / 1,461; 76.6 t/s decode, 109.4 aggregate over two streams, 691.7 t/s prefill at 166,027; MTP impossible — no head in the file | the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate | `/home/dank/models/Bonsai2-PQ2_0/` + the 10754 fork tree, `scripts/download-frank-zealot-bonsai2.sh`, revert the swap commit |
|
| 2026-10-17 → 10-18 | Ternary-Bonsai-2-**27B** PQ2_0 7,206,168,928 B `3907dc16…` + Q8_0 mmproj 629,246,976 B | PrismML fork **10754** (`~/GitPackages/llama-prism/`) | 2 × 200,704 (pool 401,408), q8_0 K+V = 13,328 MiB, settled 22,664 / 1,461; 76.6 t/s decode, 109.4 aggregate over two streams, 691.7 t/s prefill at 166,027; MTP impossible — no head in the file | the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate | `/home/dank/models/Bonsai2-PQ2_0/` + the 10754 fork tree, `scripts/download-frank-zealot-bonsai2.sh`, revert the swap commit |
|
||||||
| 2026-10-07 → 10-17 | Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K 7,359,259,008 B `c0ba7beb…` + BF16 mmproj 921,704,448 B | llama.cpp master `5fc4f3c` CUDA sm_86 (`~/GitPackages/llama-master/build-cuda/`) | 2 × 200,704, exact f16 K+V, MTP off (no `nextn` tensors) | the ternary 27B bought a larger model class at the same window | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/` + the master build, `scripts/download-frank-zealot.sh` |
|
| 2026-10-07 → 10-17 | Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K 7,359,259,008 B `c0ba7beb…` + BF16 mmproj 921,704,448 B | llama.cpp master `5fc4f3c` CUDA sm_86 (`~/GitPackages/llama-master/build-cuda/`) | 2 × 200,704, exact f16 K+V, MTP off (no `nextn` tensors) | the ternary 27B bought a larger model class at the same window | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/` + the master build, `scripts/download-frank-zealot.sh` |
|
||||||
| 2026-10-07 (same day) | Gemma4-26B-A4B-QAT Q4_K_M 16,796,015,520 B + BF16 mmproj 1,194,827,776 B + unused `gemma4-assistant` MTP 251,937,728 B | llama.cpp master CUDA | 1 × 200,704, f16 K + q8_0 V, ubatch 1024, MTP off; 21,868 / 2,257 settled, 139.9 t/s decode (the ~4 B-active MoE is at home here); MTP ON would cost ~1.9 GiB and boot at 319 MiB free | ONE slot serialized compaction; the 9B's lighter weights bought two slots and exact KV | `/home/dank/models/Gemma4-26B-A4B/` (ungated — plain `curl -fL`) |
|
| 2026-10-07 (same day) | Gemma4-26B-A4B-QAT Q4_K_M 16,796,015,520 B + BF16 mmproj 1,194,827,776 B + unused `gemma4-assistant` MTP 251,937,728 B | llama.cpp master CUDA | 1 × 200,704, f16 K + q8_0 V, ubatch 1024, MTP off; 21,868 / 2,257 settled, 139.9 t/s decode (the ~4 B-active MoE is at home here); MTP ON would cost ~1.9 GiB and boot at 319 MiB free | ONE slot serialized compaction; the 9B's lighter weights bought two slots and exact KV | `/home/dank/models/Gemma4-26B-A4B/` (ungated — plain `curl -fL`) |
|
||||||
@@ -146,12 +145,10 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
|
|||||||
3 cells, and an armed depth-2 drafter triples it to 452.25), compute buffers at graph capture, the mmproj after the
|
3 cells, and an armed depth-2 drafter triples it to 452.25), compute buffers at graph capture, the mmproj after the
|
||||||
fit. On gemma4-class hybrids `attention_k_eq_v` does **not** shrink the allocation — the cache reserves the K/V pair
|
fit. On gemma4-class hybrids `attention_k_eq_v` does **not** shrink the allocation — the cache reserves the K/V pair
|
||||||
twice, 6.125 B/elem at f16 K + q8_0 V.
|
twice, 6.125 B/elem at f16 K + q8_0 V.
|
||||||
- **`--flash-attn on` is pinned, not decorative**: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has **no true fp8 KV dtype** — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). `GGML_CUDA_FA_ALL_QUANTS` is **deprecated upstream and inert in both trees on this host** (no reference anywhere in `ggml/src/ggml-cuda/`, read 2026-10-19): master's `ggml/cmake/common.cmake` warns `use GGML_CUDA_FA_QUANTS=all instead`, and the successor's default — `GGML_CUDA_FA_QUANTS="q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16"` (`ggml/CMakeLists.txt:208`) — already compiles the quantized-KV kernels with the old flag OFF (master's `build-cuda` cache says `FA_ALL_QUANTS:BOOL=OFF` and it dispatched q8_0/q4_0 KV regardless). Prove a build's quantized-KV ability from its `CMakeCache.txt` `GGML_CUDA_FA_QUANTS` line, never from the retired flag or the build command you remember.
|
- **`--flash-attn on` is pinned, not decorative**: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has **no true fp8 KV dtype** — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). `GGML_CUDA_FA_ALL_QUANTS` is vestigial and build-specific: the PrismML fork carries it ON (since 2026-09-17) while the master build had it OFF and still dispatched Q4/Q5 KV unconditionally (`ggml-cuda/fattn.cu` has no guard), so quantized KV never required it.
|
||||||
- **Prices come from boots, not formulas, and an arm belongs to the payload it was measured on**: 3 × 180,224 +
|
- **Prices come from boots, not formulas, and an arm belongs to the payload it was measured on**: 3 × 180,224 +
|
||||||
q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does **not** fit with Q6_K inside the 2–3 GiB band.
|
q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does **not** fit with Q6_K inside the 2–3 GiB band. The
|
||||||
The mmproj worst case is printed before fitting (`estimated worst-case memory usage of mmproj is
|
mmproj worst case (~870 MiB) is printed before fitting but never billed by the engine's own fit check. Only
|
||||||
873.10 MiB` on the Bonsai-2 27B boot) but never billed by the engine's own fit check: that boot
|
|
||||||
projected **21,439 MiB** and settled at **22,664**. Only
|
|
||||||
`tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
|
`tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
|
||||||
green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
|
green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
|
||||||
projections drift.
|
projections drift.
|
||||||
@@ -202,10 +199,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
|
|||||||
`predicted_per_second`). Keep `VERBOSITY=4` — 3 is the engine default and cannot prove the KV or draft sizes,
|
`predicted_per_second`). Keep `VERBOSITY=4` — 3 is the engine default and cannot prove the KV or draft sizes,
|
||||||
while levels above 4 flood the journal.
|
while levels above 4 flood the journal.
|
||||||
- llama.cpp does **not** validate the requested model id: it answers and echoes the served `ALIAS`, so a rename breaks id-keyed consumers (the OpenWebUI `model` row, its `access_grant`, `openai.api_configs`) and nothing server-side. Alias history: `frank-small` → `frank-secondary` (2026-10-06) → `frank-zealot` (`7c77326`, 2026-10-09); the service and unit took the name 2026-10-10 (`55f87fc`).
|
- llama.cpp does **not** validate the requested model id: it answers and echoes the served `ALIAS`, so a rename breaks id-keyed consumers (the OpenWebUI `model` row, its `access_grant`, `openai.api_configs`) and nothing server-side. Alias history: `frank-small` → `frank-secondary` (2026-10-06) → `frank-zealot` (`7c77326`, 2026-10-09); the service and unit took the name 2026-10-10 (`55f87fc`).
|
||||||
- The first request after a boot is a warmup artifact, never a measurement: on the Bonsai-2 27B
|
- The first request after a boot is a warmup artifact (37 t/s class) and is never a measurement; never time a wall-clock aggregate on a generation that stops on its own (use `ignore_eos`).
|
||||||
boot request #1 gave 2.6 t/s prefill / 37.7 t/s decode and request #2 gave 1,258 t/s prefill /
|
|
||||||
54.7 t/s per stream (two concurrent, the 109.4 aggregate in the Eras row) — a 3x spread on one
|
|
||||||
shape. Never time a wall-clock aggregate on a generation that stops on its own (use `ignore_eos`).
|
|
||||||
|
|
||||||
## Files
|
## Files
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: research
|
||||||
title: FreeToken Benchmark Sweep — Dankserver RTX 3090
|
title: FreeToken Benchmark Sweep — Dankserver RTX 3090
|
||||||
description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime."
|
description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2.
|
||||||
tags:
|
tags:
|
||||||
- freetoken
|
- freetoken
|
||||||
- benchmark
|
- benchmark
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
|
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
|
||||||
description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required."
|
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
|
||||||
tags:
|
tags:
|
||||||
- ikllama
|
- ikllama
|
||||||
- llamacpp
|
- llamacpp
|
||||||
@@ -11,8 +11,7 @@ tags:
|
|||||||
- ornith
|
- ornith
|
||||||
- kv-cache
|
- kv-cache
|
||||||
- hadamard
|
- hadamard
|
||||||
- retired-endpoint
|
timestamp: 2026-08-26
|
||||||
timestamp: 2026-10-19
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
|
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
|
||||||
|
|||||||
@@ -9,33 +9,37 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|
|||||||
|
|
||||||
| Page | Owns |
|
| Page | Owns |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
|
| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
|
||||||
| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
|
| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
|
||||||
| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |
|
| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
|
||||||
|
|
||||||
## Retired endpoints
|
## Retired endpoints
|
||||||
|
|
||||||
| Page | Owns |
|
| Page | Owns |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
|
| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
|
||||||
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
|
| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
|
||||||
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
|
| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
|
||||||
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
|
| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
|
||||||
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
|
| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
|
||||||
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
|
| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
|
||||||
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
|
| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
|
||||||
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
|
| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
|
||||||
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
|
| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
|
||||||
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |
|
| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
|
||||||
|
|
||||||
## Pipeline Parallelism (series)
|
## Pipeline Parallelism (series)
|
||||||
|
|
||||||
| Page | Owns |
|
| Page | Owns |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
|
||||||
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
|
| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
|
||||||
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
|
| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
|
||||||
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |
|
| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
|
||||||
|
| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
|
||||||
|
| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
|
||||||
|
| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
|
||||||
|
| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
|
||||||
|
|
||||||
## Techniques, hardware, trials
|
## Techniques, hardware, trials
|
||||||
|
|
||||||
@@ -43,48 +47,43 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|
|||||||
|---|---|
|
|---|---|
|
||||||
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
|
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
|
||||||
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
|
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
|
||||||
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
|
| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
|
||||||
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
|
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
|
||||||
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
|
| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
|
||||||
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
|
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
|
||||||
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
|
| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
|
||||||
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
|
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
|
||||||
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
|
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
|
||||||
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
|
| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
|
||||||
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
|
| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
|
||||||
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
|
| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
|
||||||
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
|
| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
|
||||||
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
|
| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
|
||||||
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
|
| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
|
||||||
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
|
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
|
||||||
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
|
| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
|
||||||
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
|
| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
|
||||||
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
|
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
|
||||||
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
|
| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
|
||||||
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
|
| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
|
||||||
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
|
| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
|
||||||
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
|
| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
|
||||||
|
| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
|
||||||
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
|
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
|
||||||
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
|
| [[Quantization Production]] | — |
|
||||||
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
|
| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
|
||||||
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
|
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
|
||||||
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
|
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
|
||||||
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
|
| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
|
||||||
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
|
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
|
||||||
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
|
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
|
||||||
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
|
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
|
||||||
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
|
| [[swift15-campaign-worklog]] | — |
|
||||||
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
|
| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
|
||||||
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
|
| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
|
||||||
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
|
| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
|
||||||
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
|
| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
|
||||||
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
|
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
|
||||||
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
|
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
|
||||||
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
|
| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
|
||||||
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
|
| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
|
||||||
|
|
||||||
## Pages
|
|
||||||
|
|
||||||
| Page | Owns |
|
|
||||||
|---|---|
|
|
||||||
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |
|
|
||||||
@@ -1,11 +1,10 @@
|
|||||||
---
|
---
|
||||||
type: reference
|
type: reference
|
||||||
description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it."
|
description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw."
|
||||||
tags:
|
tags:
|
||||||
- llm
|
- llm
|
||||||
- rocm
|
- rocm
|
||||||
- frank
|
- frank
|
||||||
- retired-endpoint
|
|
||||||
timestamp: 2026-09-15
|
timestamp: 2026-09-15
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -240,4 +239,3 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
|
|||||||
`gl=45`. See the `set_rows` build bug section above.
|
`gl=45`. See the `set_rows` build bug section above.
|
||||||
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
|
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
|
||||||
retired the same day for the AGENT model). See [[frank-dragoon]].
|
retired the same day for the AGENT model). See [[frank-dragoon]].
|
||||||
---
|
|
||||||
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user