Compare commits
54
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
861920797d | ||
|
|
3ef711f146 | ||
|
|
823fd52d0e | ||
|
|
bda011d26f | ||
|
|
b19f1c49d6 | ||
|
|
b1ca4e02fa | ||
|
|
7f65cf0252 | ||
|
|
346e1d22f4 | ||
|
|
53ef121180 | ||
|
|
fa093e617d | ||
|
|
6215f1a6c8 | ||
|
|
8bac97ec87 | ||
|
|
69739237c8 | ||
|
|
e5580abb6f | ||
|
|
ee6d8610b3 | ||
|
|
933420fd29 | ||
|
|
9ffa99902d | ||
|
|
b8f564f3bd | ||
|
|
d42d053abe | ||
|
|
7eed636bd1 | ||
|
|
d9b82b372e | ||
|
|
3d10a4ca4f | ||
|
|
e5b731216c | ||
|
|
ce8d82d4a7 | ||
|
|
02d04bc1f6 | ||
|
|
428ac22736 | ||
|
|
5d5cac4da3 | ||
|
|
3eaba57925 | ||
|
|
9b9c32005f | ||
|
|
ea2fc6440b | ||
|
|
6248f02ca2 | ||
|
|
25df9d05a2 | ||
|
|
7601edfe28 | ||
|
|
e133d2412d | ||
|
|
9971cb559e | ||
|
|
065d796d73 | ||
|
|
1e6b39c382 | ||
|
|
1e52e45fae | ||
|
|
88342c144b | ||
|
|
d09d2745cf | ||
|
|
a3bce7d123 | ||
|
|
327e78b950 | ||
|
|
8775e64ed0 | ||
|
|
52dbe18486 | ||
|
|
2a12487653 | ||
|
|
a483e51804 | ||
|
|
c860fb3c8c | ||
|
|
fb8862b24c | ||
|
|
293c25c50b | ||
|
|
483fd843b7 | ||
|
|
a5fe045b58 | ||
|
|
3880503057 | ||
|
|
e3a02e25c7 | ||
|
|
1ed7f0ddf2 |
@@ -27,7 +27,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
|--------|---------|---------|---------|------|
|
||||
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
|
||||
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
|
||||
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj | 7771 |
|
||||
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
|
||||
|
||||
Payload identity, shape, eras, laws and rollback live in ONE page each:
|
||||
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
|
||||
@@ -47,7 +47,7 @@ per attempt and no separate swap-history file.
|
||||
The managed skill library is distilled there one page per skill-cluster:
|
||||
`LLM/index.md` → *Cross-Cutting References*, plus `Operations/index.md`,
|
||||
`Agent/index.md`, and the Infrastructure/Dankclaw/FrankBook-Pro service
|
||||
pages (`serverAppConfigs-docs/log.md` carries the wave plan and results)
|
||||
pages. There is no `log.md`: the change diary was destroyed 2026-10-19
|
||||
- `tests/` — bats suite + Playwright e2e
|
||||
|
||||
## Git Workflow — trunk-based (fleet law, aligned 2026-10-18)
|
||||
@@ -318,35 +318,36 @@ the next), so never report request #1.
|
||||
deletes the unit file from `/etc/systemd/system/`, not just the symlink.
|
||||
`mask` symlinks to `/dev/null` and is safe/reversible.
|
||||
|
||||
## Documentation Policy (fleet law, 2026-10-18)
|
||||
## Documentation Policy (fleet law, 2026-10-19)
|
||||
|
||||
**One fact, one owner.** A page owns exactly one thing — an endpoint, a
|
||||
service, or a technique — and every other file links to it with `[[Page]]`.
|
||||
Restating a shape, a digest, a port list, or a measured number in a second
|
||||
file is a defect: it is the next thing to go stale.
|
||||
**Documentation exists for point of use.** A page earns its bytes only if it
|
||||
helps whoever performs the next task — the shape to deploy, the flag that must
|
||||
be passed, the failure signature to recognise, the rollback to run. It is not a
|
||||
lab notebook and not a ledger. **Losing information is an acceptable, usually
|
||||
correct outcome.** Never let an orphan census ("this is the only copy of that
|
||||
fact") decide what to keep: sole custody of a useless fact is still useless.
|
||||
|
||||
- **One page per endpoint, forever:** `serverAppConfigs-docs/LLM/<endpoint>.md`
|
||||
— current shape, an **Eras** table (`window | payload | runtime | shape |
|
||||
replaced because | rollback`), the **Laws** the endpoint taught, the wire
|
||||
contract, and the files that own it. A payload swap edits the shape block
|
||||
and appends one Era row. It never creates a page, and the retired era's
|
||||
page goes to `.AGENTTRASH/` once its rollback path and its laws are in the
|
||||
table.
|
||||
- **`log.md` records incidents and laws** — anything with a causal story
|
||||
(why the host went down, why a knob resets at reboot, why the RAM was the
|
||||
fault). A model swap is not an incident: it is an Era row and nothing else.
|
||||
- **`AGENTS.md` carries operator-critical numbers only** — port, context
|
||||
ceiling, slot shape, unit/env/serve trio — plus the cross-cutting laws.
|
||||
- **The purpose test — before you write, and again before you keep:** would the
|
||||
reader act differently because of this line? No → delete it.
|
||||
- **Measurements are disposable.** A rate or throughput figure (t/s, tok/s, ms,
|
||||
acceptance %, a bench ladder, a measured MiB) for a payload, runtime, or
|
||||
layout that is not running now and not being compared now is destroyed, not
|
||||
filed — not in an Eras row, not on a technique page, not "preserved for
|
||||
history". AI moves faster than this vault: the number is wrong before anyone
|
||||
reads it, and a wrong number in a runbook gets believed. Keep the **law** the
|
||||
measurement taught; drop the measurement.
|
||||
- **One fact, one owner**, every other file links with `[[Page]]`. The env file
|
||||
beside the serve script is the authority for every live number; the endpoint
|
||||
page is the authority for the shape and the wire contract.
|
||||
- **One page per endpoint:** current shape, **Eras** (`window | payload |
|
||||
runtime | replaced because | rollback` — the rollback is a commit, not a
|
||||
number), the **Laws**, the wire contract, the files that own it. A swap edits
|
||||
the shape block and appends one row; the retired era's own page is
|
||||
**destroyed** to `.AGENTTRASH/`, never merged into anything.
|
||||
- **`log.md` is retired** (2026-10-19: 2,380 lines destroyed). Git holds the
|
||||
chronology. A causal law belongs to the page that owns the thing it governs,
|
||||
or to *Key Lessons Learned* below if it is cross-cutting — never to a diary.
|
||||
- **Index pages are locators:** one row, one clause, no numbers.
|
||||
- **A measured number is written once:** in the env-file header beside the
|
||||
serve script that consumes it, and in the endpoint page. Every other page
|
||||
cites it by link.
|
||||
- Technique pages (spec decode, KV math, offload budgets, PP bandwidth) stay
|
||||
technique pages: they stop carrying a per-endpoint throughput table or a live
|
||||
shape, but a campaign ledger that exists nowhere else is owned by the page that
|
||||
measured it. The defect is two pages carrying one figure, not a number living on a
|
||||
technique page — move a figure only when an endpoint page or an env header is its
|
||||
home; otherwise leave it in place and name the era it measured.
|
||||
|
||||
## Model Config (harness)
|
||||
|
||||
@@ -439,7 +440,11 @@ disagrees with it is the bug.
|
||||
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
|
||||
loader ignoring that metadata dequantizes wrong); payload
|
||||
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
|
||||
tower can crash the server with concurrent image requests).
|
||||
tower can crash the server with concurrent image requests) + the rank-1
|
||||
refusal-direction adapter applied at load by `--lora-scaled …:1`. The
|
||||
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
|
||||
— and build 10777 is also the gate that lets it load: older forks refuse it
|
||||
with `unsupported prism.hadamard.version: 2`.
|
||||
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
|
||||
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
|
||||
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
|
||||
@@ -457,12 +462,15 @@ disagrees with it is the bug.
|
||||
- Owns: `scripts/frank-archon.env`, `scripts/serve-frank-archon.sh`,
|
||||
`scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
|
||||
`scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
|
||||
`stilldeadcode/radiance@sha256:4c3f4845…`; container name = unit name.
|
||||
- Shape: TP=2 over `wht6`, `--max-num-seqs 3`, `--max-model-len 200000`,
|
||||
`expert_tiered` placement, 44,928 MiB pinned host pool per rank, VRAM pair
|
||||
`--vram-weights-mib 25600 --vram-kv-mib 5120` (ONE budget: weights + kv ≤
|
||||
30,969 MiB/card), `fp8` KV, MTP depth 2, MNBT 2048, prefix cache 8 GiB host
|
||||
+ 64 GiB disk, n-gram table pinned to RAM.
|
||||
`stilldeadcode/radiance@sha256:d590e300…` (radiance 1.8.1); container name = unit name.
|
||||
- Shape: TP=2 over `wht6`, `--max-num-seqs 2`, `--max-model-len 200000`,
|
||||
`expert_tiered` placement, 44,928 MiB pinned host pool per rank, `fp8` KV,
|
||||
MTP depth 2, MNBT 2048, `--busy-prefill-chunk auto`, prefix cache 8 GiB host
|
||||
tier only, n-gram table pinned to RAM. The per-card weights/KV split is derived
|
||||
by the engine's budget resolver from `--gpu-headroom-mib`, the cache bound and
|
||||
the host pool; the `--vram-weights-mib`/`--vram-kv-mib` pair and the prefix disk
|
||||
tier are gone (removed at 1.3.0 / 1.8.1) and naming either is a boot refusal,
|
||||
not a no-op.
|
||||
- Harness/gate contract: `models.yml` `contextWindow` MUST quote 200,000
|
||||
(over-length is refused, never truncated); the ~147 GiB pinned-RAM ledger
|
||||
is why this is a root unit, and `ttm-preflight.sh` raises
|
||||
@@ -477,481 +485,302 @@ linked runbooks.
|
||||
|
||||
**PCIe / multi-GPU**
|
||||
|
||||
- **RCCL collectives need PCIe atomics on every participating GPU.** RCCL's
|
||||
kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch
|
||||
them when the device has no atomics, and the failure surfaces as
|
||||
`hipErrorIllegalState` ("the operation cannot be performed in the present
|
||||
state") from `hipLaunchKernel`, not as anything that names atomics. The
|
||||
value is consumed at driver probe time, so writing the AtomicOp Requester
|
||||
bit with `setpci` at runtime changes nothing. Check the *chain*:
|
||||
`lspci -vv -s <gpu>` and every upstream bridge must show
|
||||
`AtomicOpsCap: Routing+` (a `Routing-` bridge is a hardware wall), and the
|
||||
endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
|
||||
6 set. AMD desktop platforms put one card on CPU root ports (atomics work)
|
||||
and the rest behind the chipset (usually not) — that asymmetry is what
|
||||
blocked frank-medium's TP=2. A one-card collective proves nothing: NCCL
|
||||
short-circuits at `world_size=1`.
|
||||
- **That wall was physical, and it was fixed on 2026-09-20 by a board swap.**
|
||||
Both paths that put two cards on CPU root ports were: an M.2→PCIe x4 riser
|
||||
into a *CPU-attached* M.2 slot, or a board whose CPU lanes bifurcate to
|
||||
x8/x8. dankserver took the second: **ASUS ProArt B850-Creator WiFi NEO**,
|
||||
`PCIEX16_1` + `PCIEX16_2` at x8/x8 Gen5 on root ports `00:01.1` / `00:01.3`,
|
||||
every bridge now `Routing+` and **both endpoints `ReqEn+`** — the exact
|
||||
precondition the old test plan named. Bandwidth was never the constraint (a
|
||||
27B TP=2 allreduce at batch 4 is ≈10 MB per token-step, ≈1.4 ms at Gen4 x4
|
||||
and under 0.4 ms at Gen5 x8).
|
||||
**Proven on the same day.** vLLM's R4D one-shot allreduce runs TP=2 here and
|
||||
frank-medium serves on it; llama.cpp's RCCL path was built too
|
||||
(`GGML_HIP_RCCL=ON`) and its tensor mode measured — real collective, slower
|
||||
shape. Note the collective choice matters: TheRock's RCCL builds lack gfx1201
|
||||
device code (`invalid device function`), which is why R4D is the path vLLM
|
||||
uses at all. Evidence matrix and the measured three-arm verdict:
|
||||
[[Dual-R9700-Cooperation]].
|
||||
- **When a GPU collective dies, read the runtime, not the framework.** The
|
||||
message that identified the cause came from `AMD_LOG_LEVEL=3`
|
||||
(`Pcie atomics not enabled, hostcall not supported` → `AQL dispatch
|
||||
failed!`), not from vLLM or RCCL. `AMD_LOG_LEVEL` costs one env var and
|
||||
prints the failing HIP call with its return code.
|
||||
- **The software path that puts both cards on ONE model without a collective:
|
||||
llama.cpp's layer split.** On the old board vLLM could not do it in any of its
|
||||
parallelisms — TP needs an allreduce, PP needs the same p2p primitive, and both
|
||||
died on the card that could not dispatch an RCCL kernel. `--split-mode
|
||||
layer` in llama.cpp needs no
|
||||
collective: layers are distributed in order and only the boundary activation
|
||||
crosses PCIe, once per forward pass. Measured here: 1991 t/s prefill /
|
||||
18.8 tok/s decode layer-split, against 272 / 26.3 tensor (butterfly
|
||||
fallback).
|
||||
This was what `frank-medium` ran until 2026-09-20, when the board swap made
|
||||
collectives legal and radiance TP=2 beat it on every axis (95.8 vs 45.7 t/s
|
||||
decode, 0.21 vs 0.72 s TTFT, 259 vs 72 t/s burst-4) — see
|
||||
[[frank-medium-radiance-mxfp4]]; the llama shape is the rollback target
|
||||
[[frank-medium-llamacpp-dflash2]].
|
||||
- **Layer split is also the cross-vendor answer — and `-ot` is what cannot be
|
||||
combined with it.** On Dankclaw (2026-09-16) a CUDA 3090 + Vulkan iGPU layer
|
||||
split ran a real two-device graph: `-dev CUDA0/Vulkan1 -sm layer -ts 0.15/0.85`
|
||||
held 14,994 MiB on the 3090 and gave 27.2 tok/s with 268.5 t/s prefill at 64k
|
||||
depth (iGPU alone: 25.8 / 231.8). What does NOT work is per-tensor placement
|
||||
across devices: `--override-tensor <group>=<device>` sets
|
||||
`has_tensor_overrides()`, a REQUIRED term of the scheduler's
|
||||
`pipeline_parallel` flag (llama-context.cpp:433 — the only place in the tree
|
||||
that reads it), and that flag is what sets `n_copies` in
|
||||
`ggml_backend_sched_new`; with `n_copies == 1` a pre-allocated weight on a
|
||||
non-primary device has no backend and the graph aborts (`ggml-backend.cpp:941`,
|
||||
reported with the leaf's own op name `(NONE)`). `=CPU` is the only override
|
||||
target that works, because the CPU backend is last in the list. Pure Vulkan
|
||||
changes nothing here — the gate is the override, not the driver mix. See
|
||||
[[x4-pair-experiment]].
|
||||
- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design
|
||||
constraint, not a detail.** `pcie-link-test.cu` on Dankclaw (2026-09-16,
|
||||
services stopped): pinned H2D 6.72 GB/s, D2H 6.60 GB/s — 85 % of the Gen4 x4
|
||||
ceiling — with async no better than sync and pageable within 1 % of pinned,
|
||||
i.e. the copies are link-limited rather than staging-limited (the usual
|
||||
"pageable costs 2x" rule does not apply here). The card is x16-capable and the
|
||||
Strix Halo port beneath it is x8-capable (`LnkCap: 16GT/s, Width x8` on
|
||||
`00:03.1`), so the x4 lives in the adapter or cable; the link idles at
|
||||
2.5 GT/s x4 and trains up under load. Consequence for any engine that
|
||||
*streams* weights across it: a 3 MiB slab costs 474 us, and a 111 GiB A3B
|
||||
model needs ~1.70 GiB of active experts per token ⇒ **3.7 tok/s** (4.6 with a
|
||||
20 % resident cache), against **20.46 tok/s** for the same model with the
|
||||
experts *computed* in host RAM by llama.cpp (`-ot exps=CPU`, whence the link
|
||||
carries activations only) and 25–26.5 on the iGPU. Streaming over PCIe loses;
|
||||
computing in RAM does not care. FreeToken — expert LRU in host RAM, exactly
|
||||
this streaming shape — also loads only modelopt block-FP8 safetensors for
|
||||
`qwen3_5_moe`, so a Q4_K_M GGUF is unloadable there before bandwidth is even
|
||||
the question. See [[x4-pair-experiment]].
|
||||
- **RCCL collectives need PCIe atomics on every participating GPU, and the failure never names
|
||||
atomics.** The kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch them on
|
||||
a device without atomics and the error surfaces as `hipErrorIllegalState` from `hipLaunchKernel`,
|
||||
not as anything that names atomics. The value is consumed at driver PROBE time, so writing the
|
||||
AtomicOp Requester bit with `setpci` at runtime changes nothing. Check the CHAIN: `lspci -vv -s
|
||||
<gpu>` and every upstream bridge must show `AtomicOpsCap: Routing+` (a `Routing-` bridge is a
|
||||
hardware wall), and the endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
|
||||
6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
|
||||
chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
|
||||
nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
|
||||
- **When a GPU collective dies, read the runtime, not the framework.** The message that identified
|
||||
the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
|
||||
`AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
|
||||
failing HIP call with its return code. [[PCIe Link and Atomics]]
|
||||
|
||||
|
||||
**ROCm / GPU**
|
||||
|
||||
- **Layer split needs no collective** — layers go in order and only the boundary
|
||||
activation crosses PCIe, so it is the fallback wherever a collective cannot be
|
||||
dispatched (and on an RCCL-less fork it beats tensor mode, which falls back to
|
||||
a butterfly allreduce). Pin the spec-decode drafter to one device: an unpinned
|
||||
draft under tensor mode aborts the KV build, and on a split its home is the
|
||||
card holding the target's last layers. Deployed history:
|
||||
[[frank-medium-llamacpp-dflash2]].
|
||||
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a
|
||||
trap.** With a missing `DST`, cp creates `DST` as a copy of `SRC` and drops
|
||||
one directory level. rocBLAS and hipBLASLt look for their device code at a
|
||||
fixed depth (`../lib/rocblas/library`, `../lib/hipblaslt/library/<arch>`),
|
||||
and the flattened copy fails at *request* time, not load time: the engine
|
||||
reads the weights, answers `/health` and dies on the first completion with
|
||||
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch
|
||||
: gfx1201`. Declare the destination, then copy the contents.
|
||||
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the
|
||||
~50 GiB PLE table into anonymous memory → silent cgroup kill, dmesg is the
|
||||
only evidence. mmap pages ARE page cache. Never pin the PLE tensor with
|
||||
`-ot` (23 → 13.4 t/s).
|
||||
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve
|
||||
scripts export env files with `set -a`, so a tunable named `TEMP` leaks
|
||||
into HIP (`TEMP=1.0` sent the JIT cache to path `1.0` and segfaulted every
|
||||
load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a tunable
|
||||
these; check with
|
||||
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a trap.** With a missing
|
||||
`DST`, cp creates `DST` as a copy of `SRC` and drops one directory level. rocBLAS and hipBLASLt
|
||||
look for their device code at a FIXED depth (`../lib/rocblas/library`,
|
||||
`../lib/hipblaslt/library/<arch>`), so the flattened copy fails at REQUEST time, not load time:
|
||||
the engine reads the weights, answers `/health` and dies on the first completion with
|
||||
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch : gfx1201`. Declare
|
||||
the destination, then copy the contents.
|
||||
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
|
||||
anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
|
||||
and pinning the PLE tensor with `-ot` was slower than letting it page. [[Llama Cpp Serving Sizing]]
|
||||
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
|
||||
with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
|
||||
`1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
|
||||
tunable these; check with
|
||||
`strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
|
||||
- **`-DGGML_CUDA_FA_ALL_QUANTS=ON` is required** for q8_0 KV + flash
|
||||
attention on CUDA builds; without it prefill collapses ~12x (95 vs 1300+
|
||||
t/s, measured 2026-08-17).
|
||||
- **Flash-Next MTP heads do not load on any current loader** — they need full
|
||||
per-layer `blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified
|
||||
3x). Don't re-attempt without new loader support.
|
||||
- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on
|
||||
qwen4exp needs `scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on
|
||||
the agentionai FAST layer set; the kingjones STRIX set works unpatched.
|
||||
- **GPU budget on Dankclaw is the GTT aperture (~114 GiB), not the
|
||||
rocm-smi "dedicated" figure.** The Radeon 8060S (gfx1151) reports
|
||||
116,732 MiB (`llama-server --list-devices`;
|
||||
`gtt_total=122402963456` in `/sys/class/drm/card1/device/`). The older
|
||||
"64 GiB, SVM-backed" lesson was already stale — the CIRU Orca predecessor
|
||||
held 92.6 GiB of weights on the iGPU. `free`/`dmidecode` show the
|
||||
CPU-side carve-out — intentional, do not change.
|
||||
- **`GGML_CUDA_FA_ALL_QUANTS` is dead; do not build for it.** Verified on this host 2026-10-19:
|
||||
master's `ggml/cmake/common.cmake` warns it is deprecated in favour of `GGML_CUDA_FA_QUANTS`, the
|
||||
successor's DEFAULT pair list (`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16`) already compiles the
|
||||
quantized-KV kernels with the old flag OFF, and neither llama tree on this host references the old
|
||||
macro anywhere under `ggml/src/ggml-cuda/`. The old "~12× prefill collapse" belongs to the older
|
||||
source line. Prove a build's KV ability from its `CMakeCache.txt`. [[frank-zealot]]
|
||||
- **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
|
||||
`blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
|
||||
loader support.
|
||||
- **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
|
||||
Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
|
||||
`/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
|
||||
`free`/`dmidecode` show the CPU-side carve-out — intentional, do not change. [[Dankclaw System Specs]]
|
||||
|
||||
**Load / crash diagnosis**
|
||||
|
||||
- Model load is slow (minutes cold); health returns 503 while loading; do
|
||||
not kill the process. `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On
|
||||
halogen, the FIRST request after a (re)start can go silent ~45–60 s
|
||||
(health `engine_unresponsive`, 503s) and then self-recovers — do not
|
||||
restart over it.
|
||||
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` + kernel line via
|
||||
`journalctl -k` (segfault at address 8 in `libamdhip64` = NULL unref in
|
||||
HIP); (2) run serve script, then raw binary — different outcome isolates
|
||||
the env file; (3) `gdb -batch -ex run -ex "bt 25" --args bash
|
||||
scripts/serve-<svc>.sh` — llama-server buffers stdout and loses the log
|
||||
tail on SIGSEGV; trust gdb, not the log; (4) `amdgpu: SVM mapping failed`
|
||||
spam is a symptom, not a root cause.
|
||||
- Journal signatures: client abort logs `W srv stop: cancel task, id_task =
|
||||
N`; natural stop logs only `slot release` with `truncated = 0`. The server
|
||||
never preempts a running generation; excess tasks queue when slots are
|
||||
busy.
|
||||
- Model load is slow (minutes cold) and health returns 503 while it runs: do NOT kill the process,
|
||||
`deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On halogen the FIRST request after a (re)start
|
||||
can go silent for ~45–60 s (`engine_unresponsive`, 503s) and then self-recovers — do not restart
|
||||
over it. [[frank-dragoon]]
|
||||
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` AND the kernel line via `journalctl -k` (a
|
||||
segfault at address 8 in `libamdhip64` is a NULL unref in HIP); (2) run the serve script, then the
|
||||
raw binary — a different outcome isolates the env file; (3) `gdb -batch -ex run -ex "bt 25"
|
||||
--args bash scripts/serve-<svc>.sh`, because llama-server buffers stdout and loses the log tail on
|
||||
SIGSEGV: trust gdb, not the log; (4) `amdgpu: SVM mapping failed` spam is a symptom, not a root
|
||||
cause. [[LLM Performance Diagnosis]]
|
||||
- Journal signatures: a client abort logs `W srv stop: cancel task, id_task = N` while a natural stop
|
||||
logs only `slot release` with `truncated = 0`. The server never preempts a running generation;
|
||||
excess tasks queue when slots are busy.
|
||||
|
||||
**Host memory / swap**
|
||||
|
||||
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
||||
of the old one, or live pages must land in RAM that `MemAvailable`
|
||||
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
||||
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
|
||||
the KV pool is worse than the OOM it prevents.
|
||||
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
||||
`swapon --show` before reasoning about headroom.
|
||||
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults
|
||||
to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the
|
||||
2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB,
|
||||
frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start,
|
||||
and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in
|
||||
`activating (auto-restart)` that exits 1 in 50 ms with no container and no
|
||||
engine log is a PREFLIGHT refusal, not an engine fault. Repo law now:
|
||||
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob
|
||||
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only
|
||||
after a write the kernel rejected. Never prescribe a manual sysfs write as the
|
||||
recovery procedure for a service that can boot before a human reaches it.
|
||||
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
||||
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
||||
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
||||
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
||||
a short in-band pattern check (60 s, 8 GiB of 246 GiB) is not a stability
|
||||
proof — combined, those two instruments said "ruled out" while the RAM
|
||||
was the cause. "4800 is the JEDEC floor on 5600-rated DIMMs" is a rating,
|
||||
not a margin guarantee. When the fault is data-shaped (NaN states,
|
||||
garbage tokens) and a software revert doesn't move it, price in the RAM
|
||||
before the runtime — and remember a revert tested under the same marginal
|
||||
hardware is itself confounded. See the runbook's 2026-10-05 section.
|
||||
- **Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` of the old one**, or
|
||||
live pages must land in RAM that `MemAvailable` overstates. Dankclaw's 64 GiB `/swap.img`
|
||||
(fstab-driven) protects the co-tenants, not an endpoint with pinned weights — paging the KV pool
|
||||
is worse than the OOM it prevents. Host swap state is invisible to `deploy.sh` (no repo file):
|
||||
`swapon --show` before reasoning about headroom. [[Swap Configuration]]
|
||||
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults to half of RAM, so
|
||||
a manual raise vanished with a power failure, frank-archon's pinned-ledger preflight refused the
|
||||
start, and `Restart=always` re-ran that refusal 6,919 times in ~20 h — **a unit in
|
||||
`activating (auto-restart)` that exits 1 in 50 ms with no container and no engine log is a
|
||||
PREFLIGHT refusal, not an engine fault.** Repo law now: `scripts/ttm-preflight.sh` raises the knob
|
||||
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only after a write the
|
||||
kernel rejected. Never prescribe a manual sysfs write as the recovery procedure for a service that
|
||||
can boot before a human reaches it. [[frank-archon]]
|
||||
- **Silent RAM corruption outranks every software theory.** frank-archon's long-conversation
|
||||
repetition/garbage was marginal DDR5 at 4800 MT/s on four DIMMs, fixed by dropping to 4600.
|
||||
Consumer non-ECC raises NO MCE for a silent flip, and a short in-band pattern check (60 s over
|
||||
8 GiB of 246 GiB) is not a stability proof — both instruments said "ruled out" while the RAM was
|
||||
the cause, and "4800 is the JEDEC floor" is a rating, not a margin guarantee. When the fault is
|
||||
data-shaped (NaN states, garbage tokens) and a software revert does not move it, price in the RAM
|
||||
before the runtime; a revert tested under the same marginal hardware is itself confounded.
|
||||
[[frank-archon]]
|
||||
|
||||
**Runtime-specific guards**
|
||||
|
||||
- **Rootless podman cannot RAISE rlimits, and `--ulimit nofile=1048576`
|
||||
dies at container create, not at runtime.** runc reports
|
||||
`error setting rlimit type 7: operation not permitted` (type 7 =
|
||||
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's
|
||||
hard limit — on Dankclaw that ceiling is 524288. Inherited-equal limits
|
||||
are fine, so `memlock=-1:-1` works iff the unit grants
|
||||
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
||||
podman to runc without any raise). Upstream halogen's own run lines pass
|
||||
NO `--ulimit` at all — copy vendor run lines before adding flags
|
||||
(measured 2026-10-05 on the frank-dragoon cutover).
|
||||
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
||||
`third_party/composable_kernel/include/ck/config.h` generated manually
|
||||
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
||||
non-greedy requests and reasoning budgets. Re-test chat before production.
|
||||
- **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
|
||||
stock binaries fail the draft load with a tensor-shape mismatch.
|
||||
- **A GGUF's custom tensor type picks the runtime — read the histogram before
|
||||
promising a file to a loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is
|
||||
ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`); stock
|
||||
llama.cpp and gufo refuse it at load, so ROCmFPX was not a preference but
|
||||
the only door. Counting tensor types out of the GGUF KV section takes one
|
||||
python read and settles a runtime argument in minutes. The build identity
|
||||
matters as much as the fork: the 2026-08-25 ROCmFPX builds SIGSEGV in
|
||||
`ggml_compute_forward_set_rows` on this same file, so
|
||||
`scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA.
|
||||
- **Speculative-decode flags move as a SET.** Handing the ROCmFPX fork
|
||||
`--model-draft` with no `--spec-type` (the naive way to build a "no MTP"
|
||||
bench arm) segfaulted at load, core dumped, no diagnostic (2026-10-02). A
|
||||
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND
|
||||
`--spec-type`, or it measures a crash instead of the absence of drafting.
|
||||
- **An orphaned scratch server taxes the whole host.** A bench child left
|
||||
alive by a client-side exception kept ~26 GiB of weights resident on a
|
||||
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
||||
Record the child pid and kill it in a `finally`, then prove `free` came back
|
||||
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
||||
which also hits frank-zealot.
|
||||
- **Rootless podman cannot RAISE rlimits, so `--ulimit nofile=1048576` dies at container CREATE**,
|
||||
not at runtime: runc reports `error setting rlimit type 7: operation not permitted` (type 7 =
|
||||
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's ceiling (524288 on
|
||||
Dankclaw). Inherited-equal limits are fine, so `memlock=-1:-1` works iff the unit grants
|
||||
`LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
|
||||
any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
|
||||
before adding flags. [[frank-dragoon]]
|
||||
- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a
|
||||
file to a loader.** A non-standard tensor type in the KV section makes stock loaders refuse
|
||||
the file at load, so a fork can be the only door rather than a preference. Counting tensor
|
||||
types takes one python read and settles a runtime argument in minutes; build identity matters
|
||||
as much as the fork, so pin the commit by SHA in any build script. [[GGUF Compatibility]]
|
||||
- **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
|
||||
(the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
|
||||
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
|
||||
measures a crash instead of the absence of drafting. [[frank-zealot]]
|
||||
- **An orphaned scratch server taxes the whole host.** A bench child left alive by a client-side
|
||||
exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
|
||||
every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
|
||||
back before believing a number — and kill by pid, never `pkill -f llama-server`.
|
||||
|
||||
**llama.cpp serving**
|
||||
|
||||
- **MTP is a property of the FILE, not of the flags.** The `-mtp` GGUF build
|
||||
carries the head as `blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`);
|
||||
`--spec-type draft-mtp` then sets `load_mtp`, and the draft context runs on
|
||||
the MAIN model's weights (`shares_model`), so there is no second copy of a
|
||||
12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
||||
tensors at all, and pointing `--model-draft` at the file would load it twice.
|
||||
- **What can be chained, what cannot, and what a chain costs on a hybrid.**
|
||||
`--spec-type a,b` builds every enabled implementation and chains drafting
|
||||
through them, in a fixed internal order (ngram types first, then the
|
||||
drafters — CLI order is irrelevant). The gate is `ctx_dft`: **draftless**
|
||||
types (the ngram family) are added unconditionally, while `draft-mtp`,
|
||||
`draft-dflash`, `draft-dspark` and `draft-eagle3` all require a draft
|
||||
CONTEXT. So a chain may hold one context-backed drafter plus any number of
|
||||
ngram types — `draft-dflash,ngram-mod` is legal and measured (2.6x the
|
||||
drafter alone on prompt-echo traffic, byte-identical to it on prose at the
|
||||
fleet's 48/64 window) — but `draft-mtp,draft-dflash` dies at load: *"context
|
||||
type MTP requested but model doesn't contain MTP layers"*, because MTP
|
||||
builds its context against whatever model the draft context holds. Two
|
||||
context-backed types cannot share one process, and MTP's head only loads
|
||||
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
||||
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
|
||||
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
||||
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
||||
disabling rollback support`, and MTP verification depends on that rollback,
|
||||
so acceptance drops on ALL traffic even where the matcher fires. Measure
|
||||
chains per architecture, never by analogy.
|
||||
- **A drafter's KV scales with the TARGET's context, and its window policy
|
||||
decides the price.** The draft context is built at `llama_n_ctx(ctx_tgt)` with
|
||||
no flag to cap it, so a drafter declaring NO sliding window caches the whole
|
||||
window on `--spec-draft-device` and can OOM the first draft graph capture
|
||||
AFTER `/health` has answered. Read `sliding_window` out of the drafter's GGUF
|
||||
header before sizing, and prefer a skewed `-ts` over a context cut. Worked
|
||||
example: [[frank-medium-llamacpp-dflash2]].
|
||||
- **A draft context bills its own f16 KV over the WHOLE POOL, and on a
|
||||
single-card endpoint that squeeze can cost more than the drafter wins.** With
|
||||
`--parallel N` the drafter pays the pool, not the slot, plus its own compute
|
||||
buffers. So the MTP question is never "does the file carry the head" but "what
|
||||
do draft KV + buffers cost against what the batch knobs leave for compute" —
|
||||
a per-shape answer. Measure the arm, never the flag; the booted arm table is
|
||||
on [[frank-zealot]].
|
||||
- **A quant's scale overhead belongs in its byte rate; doubling a measured
|
||||
figure to "get the f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element
|
||||
(32 codes + one 2-byte scale per 32-block), f16 is exactly 2.0 ⇒ the ratio is
|
||||
**1.882×**, not 2×. Derive byte rates from element count × block layout, then
|
||||
check them against the boot ledger — this repo carried an invented 2× number
|
||||
for weeks. Current KV math: [[frank-zealot]].
|
||||
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is
|
||||
BACKEND-SPECIFIC — never carry it across hosts.** Its compute buffers land
|
||||
about TWICE as heavy on the card that holds the output layers and the drafter,
|
||||
so a split balanced at one batch is unbalanced at another: re-measure the
|
||||
balance after any batch change. The logical `--batch-size` is nearly free.
|
||||
A large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit
|
||||
— no totals model can see that. Ladders: [[frank-medium-llamacpp-dflash2]]
|
||||
(R9700 split) and [[frank-zealot]] (CUDA 3090).
|
||||
- **On the CUDA 3090 that same knob has no interior optimum up to 2048.** A
|
||||
prompt shorter than the ubatch cannot see the knob at all — that is the
|
||||
control that proves the knob, not the traffic, moved the number. When a bench
|
||||
IS ordered, A/B it by restarting the SERVICE through a transient
|
||||
`FRANK_ZEALOT_ENV_FILE` drop-in, never a scratch port: one card holds one
|
||||
server. Numbers: [[frank-zealot]].
|
||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
||||
size** (a DeltaNet boundary and a full-attention boundary can differ by ~1 GiB
|
||||
each with its own KV), so balance by measurement. An affine fit reproduces the
|
||||
deployed point and overstates the gaps in between.
|
||||
[[frank-medium-llamacpp-dflash2]].
|
||||
- **ngram-mod prints no startup banner.** The load log shows the drafter's
|
||||
`adding speculative implementation 'draft-dflash'` and nothing at all for the
|
||||
matcher, so a chain can look unengaged while it is working. Identify it
|
||||
behaviourally (`draft_n` > 0 only where text repeats) or by the per-task
|
||||
`draft acceptance = ... mean len = ...` line. It costs no VRAM (a 64 MiB
|
||||
host-RAM hash table) and its window decides which traffic it can see: too
|
||||
short and it proposes on ordinary prose fragments, which costs verification
|
||||
without acceptance.
|
||||
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
|
||||
per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
|
||||
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT with
|
||||
a 400 even when the pool has room, so `models.yml` `contextWindow` must follow
|
||||
every reslice. Both slot directions cost or buy window, never pool; shorter
|
||||
slots decode slightly faster (less attention to scan, one fewer recurrent
|
||||
state). Measured reslices: [[frank-zealot]].
|
||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
||||
costs ~1.5× its file size** (the encoder's activation and compute buffers ride
|
||||
with it). Subtracting two booted rows isolates it exactly; extrapolating
|
||||
weights+KV+"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7.
|
||||
Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||
`PARALLEL × 4096`. The two-boot table: [[frank-zealot]].
|
||||
- **`images` and `image_url` are NOT interchangeable: only the content part
|
||||
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
||||
model still invents an answer if the client used the legacy top-level
|
||||
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
|
||||
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
||||
failure is silent because a hallucinated color is a plausible answer. Probe
|
||||
vision with a solid color and expect an exact word.
|
||||
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
||||
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
||||
then served frank-dragoon: every tool-using agent on that endpoint was broken
|
||||
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
||||
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
||||
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
||||
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
||||
option and the same ~12x prefill collapse described in the ROCm note above.
|
||||
f16 KV needs nothing; frank-zealot ran f16 for that reason until
|
||||
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
||||
endpoint could take q8_0 KV.
|
||||
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
||||
`-dev` and `-ts` follow this: `-dev CUDA0,Vulkan1` is two *single-device*
|
||||
configurations, never one two-device run, and `-ts 0.85,0.15` is two sweep
|
||||
values each becoming `[0.85, 0, …]` / `[0.15, 0, …]`, i.e. device 0 takes
|
||||
everything. Write `-dev A/B -ts x/y`. This artifact produced a wrong fleet
|
||||
conclusion on 2026-09-16 — "CUDA + Vulkan cannot serve one model", with two
|
||||
bogus OOMs that were really "all 111 GiB on the 24 GiB card" — and the bench
|
||||
table's own `dev` column was printing the truth the whole time. Any
|
||||
placement-dependent number needs an independent check: sample the other
|
||||
- **MTP is a property of the FILE, not of the flags.** The `-mtp` export carries the head as
|
||||
`blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`); `--spec-type draft-mtp` sets
|
||||
`load_mtp` and the draft context runs on the MAIN model's weights (`shares_model`), so there
|
||||
is no second copy of a 12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
||||
tensors at all, and pointing `--model-draft` at that file loads it twice. [[frank-zealot]]
|
||||
- **What can be chained: ONE context-backed drafter plus any number of draftless ngram types.**
|
||||
`--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
|
||||
`ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
|
||||
`draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
|
||||
legal and it wins on prompt-echo traffic, but `draft-mtp,draft-dflash`
|
||||
dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
|
||||
chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
|
||||
hybrid loses, because adding the ngram layer logs
|
||||
`recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
|
||||
MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
|
||||
[[frank-dragoon]]
|
||||
- **A drafter's KV scales with the TARGET's context, and its window policy decides the price.**
|
||||
The draft context is built at `llama_n_ctx(ctx_tgt)` with no flag to cap it, so a drafter
|
||||
declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
|
||||
first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
|
||||
drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
|
||||
compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
|
||||
wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
|
||||
cost against what the batch knobs leave for compute. Measure the arm, never the flag.
|
||||
[[frank-zealot]]
|
||||
- **A quant's scale overhead belongs in its byte rate; doubling a measured figure to "get the
|
||||
f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element (32 codes + one 2-byte scale per
|
||||
32-block), f16 is exactly 2.0 ⇒ the ratio is **1.882×**, not 2×. Derive byte rates from element
|
||||
count × block layout, then check them against the boot ledger — this repo carried an invented
|
||||
2× number for weeks. Current KV math: [[frank-zealot]]
|
||||
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is BACKEND-SPECIFIC — never
|
||||
carry it across hosts.** Its compute buffers land about TWICE as heavy on the card that holds
|
||||
the output layers and the drafter, so a split balanced at one batch is unbalanced at another:
|
||||
re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
|
||||
large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
|
||||
model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
|
||||
control that proves the knob, not the traffic, moved the number. The optimum is a property of
|
||||
the backend, so re-find it after any device or payload change. When a bench IS ordered, A/B it
|
||||
by restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
|
||||
port: one card holds one server.
|
||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
|
||||
boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
|
||||
by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
|
||||
matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
|
||||
only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
|
||||
costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
|
||||
too short and it proposes on ordinary prose fragments, costing verification without acceptance.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
|
||||
the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
|
||||
prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
|
||||
`contextWindow` MUST follow every reslice. Both slot directions cost or buy window, never
|
||||
pool; shorter slots decode slightly faster. Measured reslices: [[frank-zealot]].
|
||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj costs ~1.5× its file
|
||||
size** (the encoder's activation and compute buffers ride with it). Subtracting two booted rows
|
||||
isolates it exactly; extrapolating weights+KV+"a bit for buffers" is what leaves 1.1 GiB free
|
||||
when you meant 2.7. Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||
`PARALLEL × 4096`. **The projector is NOT billed in llama.cpp's own fit check**: the engine
|
||||
prints its worst case and fits WITHOUT it, so price a shape from settled `nvidia-smi` after a
|
||||
real boot — the projector, and any drafter context, lands after `/health` would already have
|
||||
answered (same class as the DSpark drafter OOM). Two-boot table: [[frank-zealot]].
|
||||
- **`images` and `image_url` are NOT interchangeable: only the content part reaches the model.**
|
||||
`--mmproj` loads, `usage.prompt_tokens` rises, and the model still invents an answer if the
|
||||
client used the legacy top-level `images` field on `/v1/chat/completions` — frank-zealot
|
||||
answered "Brown." for a solid-red PNG that way and "Red." through the OpenAI `image_url` part.
|
||||
The failure is silent because a hallucinated color is a plausible answer; probe with a solid
|
||||
color and expect an exact word. [[frank-zealot]]
|
||||
- **`--jinja` is what `tools` needs.** A subagent died with `500 tools param requires --jinja
|
||||
flag` on the ik_llama.cpp runtime that served frank-dragoon: every tool-using agent on that
|
||||
endpoint was broken and the failure read like a model fault. Mainline llama.cpp defaults
|
||||
`--jinja` on and the serve scripts pin it; halogen also accepts `tools` and returns
|
||||
`tool_calls`. Check tool support before routing a role at an endpoint.
|
||||
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's template accepts
|
||||
only `xhigh`, `medium` and `low` for `reasoning_effort` and raises on anything else — HTTP 500,
|
||||
not a clamp — so the harness's levels must be folded onto that set in `models.yml`
|
||||
(`compat.reasoningEffortMap`). A role whose level is unmapped fails on its FIRST request and
|
||||
the failure names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||
`Unexpected reasoning effort high` until `high: xhigh` was added. Halogen accepts the wider
|
||||
set, so the two entries legitimately differ.
|
||||
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe reads as
|
||||
blindness.** A probe at `max_tokens: 64` returned `content: ""` with `finish_reason: "length"`
|
||||
and half a thought in `reasoning_content` while the image HAD reached the model (322 prompt
|
||||
tokens vs 42 for the text alone). Budget ≥512 for any thinking model you probe for vision.
|
||||
- **The FIRST request after a boot is a warmup artifact**, and the 3× spread on one shape makes
|
||||
graph-capture cost read as a model fault. Benchmark the second request or later.
|
||||
[[frank-zealot]]
|
||||
- **A fork build hides its own dtype laws.** Vendor builds carry their own compile options, so
|
||||
prove a build's KV behaviour from the KV buffer line in the boot ledger and its
|
||||
`CMakeCache.txt` — never from the build flags you remember passing. [[frank-zealot]]
|
||||
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed**: without it
|
||||
the engine answers **501 Not Implemented** on `/metrics`, so the scrape fails while `/health`
|
||||
answers 200 and `LlmEndpointDown` fires beside a healthy service. halogen serves `/metrics`
|
||||
natively, so the asymmetry reads as "only the 3090 is broken". Diagnose with `lastError` from
|
||||
`/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot
|
||||
re-stamps `startsAt` on an alert that has fired for weeks. [[Observability]] [[frank-zealot]]
|
||||
- **A leftover bench drop-in silently hijacks the next deploy, and no health gate sees it.** The
|
||||
sanctioned bench-window method (a transient `Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`
|
||||
drop-in sourcing the repo env) outlives the bench unless retired: a leftover
|
||||
`frank-zealot.service.d/ptq1.conf` made the cutover deploy boot a **different model at a
|
||||
different shape** while `/health` answered 200, `/v1/models` echoed the alias (a flag, not a
|
||||
fact) and the Playwright gate passed 33/33. After deploying any recently-benched LLM service:
|
||||
`systemctl show <svc> -p DropInPaths` (must be empty), then
|
||||
`tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger — and retire drop-ins
|
||||
to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`. [[frank-zealot]]
|
||||
- **A bench verdict that never reaches the repo is indistinguishable from a rogue arm.** That
|
||||
same arm served production for 24 minutes with real numbers (~48 t/s decode, 801 t/s prefill at
|
||||
141k) that lived only in `/tmp` env files and journald, so the next agent read the surviving
|
||||
drop-in as an unfinished experiment and reported a hijack. Write the measurement into the
|
||||
endpoint page and the arm script's header in the session that takes it — `/tmp` and the journal
|
||||
are not records.
|
||||
- **`llama-bench` separates a LIST with `/` and SWEEP values with `,`** — for both `-dev` and
|
||||
`-ts`. `-dev CUDA0,Vulkan1` is two *single-device* configurations, never one two-device run,
|
||||
and `-ts 0.85,0.15` is two sweep values each putting everything on device 0. Write
|
||||
`-dev A/B -ts x/y`. This artifact produced a wrong fleet conclusion ("CUDA + Vulkan cannot serve
|
||||
one model") from two bogus OOMs, and the bench table's own `dev` column was printing the truth
|
||||
the whole time. Any placement-dependent number needs an independent check: sample the other
|
||||
device's memory (`nvidia-smi`) while the run is loaded.
|
||||
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's
|
||||
template accepts only `xhigh`, `medium` and `low` for `reasoning_effort` and
|
||||
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
||||
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
||||
A role whose level is unmapped fails on its FIRST request, and the failure
|
||||
names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
||||
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
||||
differ.
|
||||
- **The vision projector is NOT billed in llama.cpp's own fit check.** The
|
||||
engine prints `[mtmd] estimated worst-case memory usage of mmproj is
|
||||
873.10 MiB` and then fits WITHOUT it: frank-zealot's Bonsai-2 boot projected
|
||||
21,439 MiB and settled at 22,664 (2026-10-17). Price a 3090-class shape from
|
||||
settled `nvidia-smi` after a real boot, never from
|
||||
`model + context + compute` — the projector (and any drafter context) lands
|
||||
after `/health` would already have answered. Same failure class as the
|
||||
DSpark drafter OOM.
|
||||
- **The FIRST request after a boot is a warmup artifact.** Measured on
|
||||
Bonsai-2/3090: 2.6 t/s prefill and 37.7 t/s decode on request #1, then
|
||||
1,258 t/s prefill and 54.7 t/s (per stream, two concurrent) on the next — a
|
||||
3× spread on the same shape. Benchmark the second request or later, or the
|
||||
graph-capture cost reads as a model fault.
|
||||
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe
|
||||
reads as blindness.** The Bonsai-2/`image_url` probe at `max_tokens: 64`
|
||||
returned `content: ""` with `finish_reason: "length"` and half a thought in
|
||||
`reasoning_content` — the image HAD reached the model (322 prompt tokens vs
|
||||
42 for the text alone). At 512 tokens the same image answers "Red". Budget
|
||||
≥512 for any thinking model you probe for vision.
|
||||
- **Quantized KV is not the only dtype law a fork build can hide.** Vendor
|
||||
builds carry their own compile options: frank-zealot's PrismML prebuilt
|
||||
served `--cache-type-v q8_0` under flash attention at the exact 34,816
|
||||
B/token (the `GGML_CUDA_FA_ALL_QUANTS` door), so a build decision must be
|
||||
proven by the KV buffer line in the boot ledger, not by the build flags you
|
||||
remember passing.
|
||||
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed.**
|
||||
Without it the engine answers **501 Not Implemented** on `/metrics`, so the scrape
|
||||
fails while `/health` answers 200 and `LlmEndpointDown` fires beside a healthy
|
||||
service — that is how `frank-zealot` sat red across four payload generations
|
||||
(the oldest retained sample of `up{job="frank-zealot"}`, 2026-10-11, is already
|
||||
0). halogen serves `/metrics` natively, so the asymmetry reads as "only the 3090
|
||||
is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list,
|
||||
and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an
|
||||
alert that has been firing for weeks.
|
||||
- **A leftover bench drop-in silently hijacks the next deploy, and no health
|
||||
gate sees it.** The sanctioned bench-window method — a transient
|
||||
`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env` drop-in, the arm env
|
||||
`source`ing the repo env and then overriding it — outlives the bench unless
|
||||
it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` —
|
||||
the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover
|
||||
deploy boot a **different model at a different shape** (3 slots of 133,888,
|
||||
not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias
|
||||
(a flag, not a fact) and the Playwright gate passed 33/33. After deploying any
|
||||
recently-benched LLM service: `systemctl show <svc> -p DropInPaths` (must be
|
||||
empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot
|
||||
ledger — and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` +
|
||||
`daemon-reload`.
|
||||
- **A bench verdict that never reaches the repo is indistinguishable from a
|
||||
rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real
|
||||
numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only
|
||||
in `/tmp` env files and journald, so the next agent on the host read the
|
||||
surviving drop-in as an unfinished experiment and reported a hijack. Write the
|
||||
measurement into the runbook and the arm script's header in the session that
|
||||
takes it — `/tmp` and the journal are not records.
|
||||
|
||||
**Ops**
|
||||
|
||||
- Never `pkill -f` a pattern that appears in your own command line — it kills
|
||||
your ssh session. Use `pkill -x <exact-name>`.
|
||||
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in
|
||||
committed scripts (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts
|
||||
without it.
|
||||
- ComfyUI on Dankclaw runs with **GPU access** (`/dev/kfd` + `/dev/dri` via
|
||||
`AddDevice`, `GroupAdd=keep-groups`, plus the udev rule that makes those nodes
|
||||
world-writable — user-unit spawns carry no supplementary groups). It must NOT
|
||||
pass `--cpu`: the gfx1151 ROCm wheels exist and the unit's `--cpu` era ended
|
||||
2026-09-16, when a stale device-less container was found restarted 9,146
|
||||
times (`RuntimeError: No HIP GPUs are available`).
|
||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
||||
ports off the gluetun service.
|
||||
- Dankclaw boots **`iommu=pt`** since 2026-10-18. The boot line is tracked in
|
||||
`dankclaw/host/grub` and moves ONLY through `scripts/set-dankclaw-bootline.sh`
|
||||
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by
|
||||
hand-editing `/etc/default/grub`. It replaced `amd_iommu=off` (2026-09-11),
|
||||
adopted on the vendor's 13–16 % prefill figure and never measured here: the
|
||||
local A/B on byte-identical traffic says identity mode costs **nothing**
|
||||
(+2.0 %, 1,616 t/s cold prefill), so the XDNA2 NPU returned for free —
|
||||
`amdxdna` binds `c6:00.1` (PASID width 10, identity group 26) and the old
|
||||
`err=-19` only ever asked for an enabled IOMMU. **AMD PerfOpt** (iommu tree
|
||||
2026-09-24, Linux 7.4) still skips the page walk outright for the privileged
|
||||
iGPU, but it arms only in the identity domain and 26.04 caps at `7.0.0-38`.
|
||||
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
|
||||
only the CURRENT instance; history across recreations lives in
|
||||
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
|
||||
(per-user storage — `sudo podman logs` finds nothing).
|
||||
- **A dead rootless podman pause process breaks ALL container management on the
|
||||
host, and `podman system migrate` panics in 5.7.0.** Symptoms: every `podman`
|
||||
command fails with `invalid internal status … could not find any running
|
||||
process`, every container start fails binding podman's bridge addresses
|
||||
(`Error starting server failed to bind udp listener on 10.89.x.1:53: Cannot
|
||||
assign requested address`), and every `Restart=always` quadlet loops. Container
|
||||
storage is intact; what is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
|
||||
token, not a pid) and the rootless netns holding the bridges — deleting the
|
||||
stale token is not enough, and `podman system migrate` segfaults in
|
||||
`teardownNetworkBackend`. Repair: **reboot** (`/run/user` clears and all 28
|
||||
quadlets return on their own — check `Linger=yes` and `WantedBy=default.target`
|
||||
first). Seen 2026-09-16, hours into heavy GPU work on the same host — but the
|
||||
trigger is NOT established (`journalctl -k` recorded no OOM kill; the kernel
|
||||
had no other word on it). Treat the pause process as host state that can die
|
||||
for its own reasons, and reach for the reboot when it does.
|
||||
- **A container cannot reach a `Network=host` service at the host's LAN IP.**
|
||||
netavark DNATs *published* ports only: from a bridge container
|
||||
`192.168.0.143:<published>` answers while `192.168.0.143:<host bind>` and the
|
||||
gateway `10.89.1.1` are both refused. Use `host.containers.internal`
|
||||
(169.254.1.2). This is what 502'd `hermes.franksdank.cloud` until the Caddyfile
|
||||
upstream was changed on 2026-09-16 (its vhost comment had claimed the LAN IP
|
||||
was open from that bridge — it never was).
|
||||
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as
|
||||
`.partNofM` fragments.** `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF`
|
||||
carries Q4_K_M (37.00 + 37.00 + 36.97 = 110.97 GiB) and Q4_K_S (35.00 + 35.00
|
||||
+ 34.09 = 104.09 GiB) as parts, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*,
|
||||
IQ4_XS) are single files. Fetch the parts, verify each against the Hub's LFS
|
||||
sha256, concatenate in numeric order — that is how the deployed 111 GiB file
|
||||
was assembled. Listing the Hub tree and filtering `endswith(".gguf")` makes
|
||||
Q4_K_S look like it does not exist.
|
||||
- **A fetch script must `-f` and verify, or it reports success on an access
|
||||
error.** `curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body
|
||||
as the "artifact": a gated repo yielded three 153-byte shards with identical
|
||||
sha256 values and a cheerful `DOWNLOAD_DONE` in 20 seconds for 104 GiB. Use
|
||||
`-f`/`--fail-with-body`, compare each file's byte count to the Hub's declared
|
||||
size, and check the LFS sha256 — `scripts/download-frank-medium-gguf.sh`
|
||||
already does all three; ad-hoc fetches tend not to. Gated repos (e.g.
|
||||
`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, same 104.09 GiB Q4_K_S)
|
||||
return 403 even with the host's HF token until that account accepts the gate.
|
||||
- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup
|
||||
campaign, four read-only scouts worked the same pages: one classified era pages as
|
||||
stale *because* they document retired payloads (that is an era page's content), one
|
||||
reported files that do not exist, one reported duplications that grep disproved
|
||||
(`457.9 t/s` and the ping-RTT table are solo-held by `Ethernet Bandwidth Math`),
|
||||
and one cited a line number past the end of the file it judged. Re-verify every
|
||||
claimed duplicate and every claimed staleness against the file and today's fleet
|
||||
state before editing; a scout yield buys no evidence.
|
||||
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
|
||||
Use `pkill -x <exact-name>`.
|
||||
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
|
||||
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
|
||||
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
|
||||
`GroupAdd=keep-groups`, plus the udev rule that makes those nodes world-writable — user-unit
|
||||
spawns carry no supplementary groups) and must NOT pass `--cpu`: the gfx1151 ROCm wheels exist
|
||||
and a stale device-less container was found restarted 9,146 times. [[ComfyUI]]
|
||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun
|
||||
service. [[SearXNG]] [[Gluetun]]
|
||||
- **Dankclaw's kernel command line is a repo file.** It boots **`iommu=pt`** since 2026-10-18,
|
||||
tracked in `dankclaw/host/grub`, and moves ONLY through `scripts/set-dankclaw-bootline.sh`
|
||||
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by hand-editing
|
||||
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
|
||||
says identity mode costs nothing, so the XDNA2 NPU returned for free.
|
||||
[[dankclaw-iommu-translation-off]]
|
||||
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
|
||||
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
|
||||
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
|
||||
[[Quadlet Lifecycle]]
|
||||
- **A dead rootless podman pause process breaks ALL container management on the host, and
|
||||
`podman system migrate` panics in 5.7.0.** Every `podman` command fails with
|
||||
`invalid internal status`, every start fails binding the bridges
|
||||
(`failed to bind udp listener on 10.89.x.1:53`), every `Restart=always` quadlet loops — while
|
||||
container storage stays intact. What is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
|
||||
token) and the rootless netns; deleting the stale token is not enough. Repair: **reboot**
|
||||
(`/run/user` clears and the quadlets return on their own — check `Linger=yes` and
|
||||
`WantedBy=default.target` first). The 2026-09-16 occurrence had no OOM in `journalctl -k`:
|
||||
treat the pause process as host state that can die for its own reasons. [[Quadlet Lifecycle]]
|
||||
- **A container cannot reach a `Network=host` service at the host's LAN IP**: netavark DNATs
|
||||
*published* ports only, so the host bind and the `10.89.1.1` gateway are both refused from a
|
||||
bridge container while the published port answers. Use `host.containers.internal`
|
||||
(169.254.1.2). This 502'd `hermes.franksdank.cloud` until its Caddyfile upstream changed.
|
||||
[[Container Host Reachability]]
|
||||
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as `.partNofM`
|
||||
fragments** while the smaller rungs are single files, so filtering `endswith(".gguf")` makes a
|
||||
whole quant class look absent. Fetch the parts, verify each against the Hub's LFS sha256,
|
||||
concatenate in numeric order. [[Model Swap and Rollback]]
|
||||
- **A fetch script must `-f` and verify, or it reports success on an access error.**
|
||||
`curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
|
||||
repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
|
||||
seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
|
||||
declared size, and check the LFS sha256 — the live `scripts/download-frank-*.sh` scripts do
|
||||
all three; ad-hoc fetches tend not to. Gated repos 403 even with the host's HF token until
|
||||
that account accepts the gate. [[Model Swap and Rollback]]
|
||||
- **A verdict is a lead — the scout's and the tool's alike.** In the 2026-10-19
|
||||
documentation campaign six scouts worked the same pages and produced invented
|
||||
bullet IDs, files that do not exist, "duplications" grep disproved, line
|
||||
numbers past the end of the file they judged, and Eras tables with invented
|
||||
payloads. The mechanical tool was worse: `scripts/port-census.py` built its
|
||||
corpus from the directory containing the page under test, so each page was held
|
||||
by *itself* and ten era pages reported "0 sole-record lines" — which reads as
|
||||
"pure duplication, trash it". Nothing was cut on a verdict; every holder was
|
||||
re-grepped and every tool re-measured before it was believed.
|
||||
- **Never let a census decide what to keep.** That same campaign then concluded
|
||||
the era pages were irreplaceable because each held 14–63 sole-record lines —
|
||||
wrong in the opposite direction: those were throughput figures for payloads that
|
||||
no longer run. Point of use is the only ownership test, the operator applies it,
|
||||
and 53 pages plus the 2,380-line change diary were destroyed as a result. A tool
|
||||
can say where a fact lives; it cannot say whether anything needs it.
|
||||
|
||||
## Runbook Index
|
||||
|
||||
@@ -966,12 +795,13 @@ per-attempt page and no per-swap ledger anywhere else.
|
||||
| `frank-zealot` — llama.cpp (PrismML fork), RTX 3090 | `serverAppConfigs-docs/LLM/frank-zealot.md` |
|
||||
| `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
|
||||
| Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||
| Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
|
||||
|
||||
Retired **endpoints** (not payloads) keep their own pages — `frank-medium`,
|
||||
`frank-npu`, the pre-name Dankclaw/Dankserver eras — because no live page owns
|
||||
them; `serverAppConfigs-docs/index.md` is the locator for every page in the
|
||||
vault. Retired scripts live in `old/` and `.AGENTTRASH/`.
|
||||
There is no page for a retired endpoint and no change diary. A payload swap
|
||||
appends one row to the live endpoint's **Eras** table; when the endpoint itself
|
||||
dies, its page is destroyed. Rollback is a commit, not a document.
|
||||
`serverAppConfigs-docs/index.md` is the locator for every page that survives;
|
||||
retired scripts live in `old/` and `.AGENTTRASH/`.
|
||||
|
||||
## Why
|
||||
|
||||
|
||||
@@ -162,6 +162,25 @@ marreta.franksdank.cloud {
|
||||
}
|
||||
}
|
||||
|
||||
# --- Open Notebook (guarded: Anubis → Authentik; notebook-proxy provider, internal_host caddy:8017) ---
|
||||
# The podcast generator. ONE upstream carries the whole app: the image's own
|
||||
# Next.js layer rewrites /api/* to the FastAPI beside it (INTERNAL_API_URL,
|
||||
# default http://localhost:5055 inside the container), so a browser on this
|
||||
# vhost never needs the API port. That is why scripts/open-notebook.env sets
|
||||
# API_URL to exactly this origin — left unset, the app auto-detects
|
||||
# `<host-you-opened>:5055`, and 5055 on this host belongs to seerr — and why
|
||||
# the published API port sits on loopback. App-level login
|
||||
# (OPEN_NOTEBOOK_PASSWORD) stays on behind the chain: two owners, edge identity
|
||||
# and the app's own session.
|
||||
# Dialled by published LAN port, not container name: open-notebook is a
|
||||
# rootless quadlet and this Caddy lives in root's storage, so root's DNS does
|
||||
# not see the container (the same reason marreta and SearXNG are dialled by IP).
|
||||
notebook.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
||||
pihole.franksdank.cloud {
|
||||
basicauth {
|
||||
@@ -420,6 +439,16 @@ http://:8017 {
|
||||
handle @marreta {
|
||||
reverse_proxy 192.168.0.143:8082
|
||||
}
|
||||
|
||||
@notebook host notebook.franksdank.cloud
|
||||
handle @notebook {
|
||||
# Bare proxy, like ai.franksdank.cloud: the app's SSE answers
|
||||
# (/api/search/ask, chat sessions) stream for minutes through this
|
||||
# same listener with no flush or timeout tuning, so nothing here
|
||||
# needs it — a read/write timeout on the route would be the thing
|
||||
# that cuts them.
|
||||
reverse_proxy 192.168.0.143:8502
|
||||
}
|
||||
}
|
||||
|
||||
# --- Optional: Jackett ---
|
||||
|
||||
Executable
+65
@@ -0,0 +1,65 @@
|
||||
#!/bin/sh
|
||||
# halogen-fabric-clock hold | release | status: the AMD GPU's fabric clock (FCLK), held at its top level.
|
||||
#
|
||||
# On Strix Halo the GPU and the NPU working at the same time can hang the machine, and corrupt the NPU's results,
|
||||
# while the GPU's fabric clock changes speed; held at its top level, the two run together cleanly. halogen-flash-server
|
||||
# with HALOGEN_NPU_MODELS refuses to start until the clock is held. This holds it: the performance level `manual` and
|
||||
# only the top pp_dpm_fclk level, on every amdgpu device that has the control (the GPU and SoC clocks stay automatic).
|
||||
# Cost: about 1 W at idle. Run as root; halogen-fabric-clock.service runs `hold` at every boot and after a resume.
|
||||
#
|
||||
# hold hold it (waits up to 60 s for the GPU driver to come up)
|
||||
# release give it back to the driver (the performance level `auto`)
|
||||
# status print each device's state; exit 0 when every device is held
|
||||
set -u
|
||||
SYS="${HALOGEN_SYSFS:-/sys}"
|
||||
|
||||
devices() {
|
||||
for d in "$SYS"/class/drm/card*/device; do
|
||||
[ -w "$d/pp_dpm_fclk" ] || [ -r "$d/pp_dpm_fclk" ] || continue
|
||||
[ -e "$d/power_dpm_force_performance_level" ] || continue
|
||||
echo "$d"
|
||||
done | sort -u
|
||||
}
|
||||
|
||||
held() { # held DEVICE_DIR: the performance level high, or manual with only the top fclk level selected
|
||||
perf=$(cat "$1/power_dpm_force_performance_level" 2>/dev/null)
|
||||
[ "$perf" = high ] && return 0
|
||||
[ "$perf" = manual ] || return 1
|
||||
top=$(grep . "$1/pp_dpm_fclk" | tail -n 1)
|
||||
[ "$(grep -c '\*' "$1/pp_dpm_fclk")" = 1 ] && [ "${top%\*}" != "$top" ]
|
||||
}
|
||||
|
||||
cmd="${1:-status}"
|
||||
case "$cmd" in
|
||||
hold)
|
||||
n=0
|
||||
while [ -z "$(devices)" ] && [ "$n" -lt 60 ]; do sleep 1; n=$((n + 1)); done
|
||||
[ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control (pp_dpm_fclk) under $SYS/class/drm" >&2; exit 1; }
|
||||
rc=0
|
||||
for d in $(devices); do
|
||||
lvl=$(grep . "$d/pp_dpm_fclk" | tail -n 1 | cut -d: -f1)
|
||||
# the driver applies the level several seconds after the write, so the read-back waits up to 30 s
|
||||
ok=0
|
||||
if echo manual > "$d/power_dpm_force_performance_level" && echo "$lvl" > "$d/pp_dpm_fclk"; then
|
||||
t=0; until held "$d" || [ "$t" -ge 300 ]; do sleep 0.1; t=$((t + 1)); done
|
||||
held "$d" && ok=1
|
||||
fi
|
||||
if [ "$ok" = 1 ]; then
|
||||
echo "halogen-fabric-clock: $d: held at level $lvl ($(grep . "$d/pp_dpm_fclk" | tail -n 1 | awk '{print $2}'))"
|
||||
else
|
||||
echo "halogen-fabric-clock: $d: could not hold it (level $lvl)" >&2; rc=1
|
||||
fi
|
||||
done
|
||||
exit "$rc" ;;
|
||||
release)
|
||||
for d in $(devices); do echo auto > "$d/power_dpm_force_performance_level" && echo "halogen-fabric-clock: $d: released (auto)"; done ;;
|
||||
status)
|
||||
[ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control"; exit 1; }
|
||||
rc=0
|
||||
for d in $(devices); do
|
||||
if held "$d"; then echo "$d: held ($(cat "$d/power_dpm_force_performance_level"))"
|
||||
else echo "$d: NOT held ($(cat "$d/power_dpm_force_performance_level"))"; rc=1; fi
|
||||
done
|
||||
exit "$rc" ;;
|
||||
*) echo "usage: halogen-fabric-clock hold | release | status" >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -0,0 +1,19 @@
|
||||
# Holds the AMD GPU's fabric clock at its top level at boot and after every resume, for halogen-flash-server with
|
||||
# the NPU (HALOGEN_NPU_MODELS): on Strix Halo the GPU and the NPU at once are safe only with that clock held.
|
||||
# Install (as root), from this directory:
|
||||
# install -m 755 halogen-fabric-clock /usr/local/sbin/
|
||||
# install -m 644 halogen-fabric-clock.service /etc/systemd/system/
|
||||
# systemctl daemon-reload && systemctl enable --now halogen-fabric-clock.service
|
||||
# Check: halogen-fabric-clock status. Undo: systemctl disable halogen-fabric-clock.service, then
|
||||
# halogen-fabric-clock release (or reboot). The unit runs once at boot and once after each resume (a resume can hand
|
||||
# the clock back to the driver); it holds nothing open, so it shows inactive between runs.
|
||||
[Unit]
|
||||
Description=Hold the AMD GPU fabric clock at its top level (GPU and NPU together)
|
||||
After=systemd-modules-load.service suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/halogen-fabric-clock hold
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
|
||||
@@ -48,13 +48,22 @@
|
||||
# without `podman unshare` — accepted for a datastore nobody edits by hand;
|
||||
# episode AUDIO is a host directory for the opposite reason (see the app unit).
|
||||
#
|
||||
# HEALTH. The base image is distroless: no shell, no curl, no wget, so no
|
||||
# CMD-SHELL probe and no wget/curl probe is possible here (same class of unit
|
||||
# as loki, whose probe had to move into the image). SurrealDB ships its own
|
||||
# probe instead — `surreal is-ready` connects to ws://localhost:8000 by default
|
||||
# and needs no credentials — verified against this digest: it prints `OK` and
|
||||
# exits 0 while the server is up. Argument-only HealthCmd, like every unit on
|
||||
# this host, because the quadlet generator splits it on spaces.
|
||||
# HEALTH. The base image is distroless — measured against this digest, not even
|
||||
# `ls` exists (`exec: "ls": executable file not found in $PATH`), so there is no
|
||||
# `/bin/sh`, no curl and no wget. SurrealDB ships its own probe instead:
|
||||
# `surreal is-ready` connects to ws://localhost:8000 by default, needs no
|
||||
# credentials, prints `OK` and exits 0 while the server is up.
|
||||
#
|
||||
# It must be handed over as a QUADLET-SIDE JSON ARRAY, not a bare command.
|
||||
# podman wraps a bare `HealthCmd` string as `CMD-SHELL` (measured on this host's
|
||||
# podman 5.7: `--health-cmd "/surreal is-ready"` stores
|
||||
# `{"Test":["CMD-SHELL","/surreal is-ready"]}`), and with no `/bin/sh` in the
|
||||
# image that probe fails in ~30 ms with empty output on every interval while the
|
||||
# database answers perfectly — which is exactly how this service's first deploy
|
||||
# died on 2026-10-19 (`did not become healthy in 90s`). The quoted array survives
|
||||
# the generator (`--health-cmd "[\"/surreal\",\"is-ready\"]"`) and stores
|
||||
# `["CMD","/surreal","is-ready"]`, which execs directly and goes healthy. Do not
|
||||
# "simplify" it back to a bare command.
|
||||
|
||||
[Unit]
|
||||
Description=SurrealDB (open-notebook datastore)
|
||||
@@ -70,7 +79,7 @@ EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||
Exec=start --log info rocksdb:/mydata/mydatabase.db
|
||||
Volume=open-notebook-db:/mydata
|
||||
HealthCmd=/surreal is-ready
|
||||
HealthCmd=["/surreal","is-ready"]
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
|
||||
@@ -33,17 +33,21 @@
|
||||
# and 5055 is exactly the port seerr already holds on this host
|
||||
# (`PublishPort=127.0.0.1:5055:5055`; `ss -ltn` on Dankclaw 2026-10-18 shows
|
||||
# 127.0.0.1:5055 taken). So the API leaves the container on host port 5056.
|
||||
# That forces one setting: the frontend auto-detects the API as
|
||||
# `<host-you-opened>:5055`, so API_URL in scripts/open-notebook.env must name
|
||||
# the remapped port explicitly, or the UI loads and every call fails. Both
|
||||
# ports are published on all interfaces because the consumer is a browser on
|
||||
# the LAN — the fleet's browser-facing services (open-webui, comfyui, gitea)
|
||||
# bind the same way, while loopback-only is reserved for services whose user
|
||||
# path is the guard chain. This service has NO vhost and no Caddy route: it is
|
||||
# internal-only until it is guarded, and its own password
|
||||
# (OPEN_NOTEBOOK_PASSWORD) is what stands in front of it. That password is not
|
||||
# decoration: with it unset the API answers everyone on the LAN, including the
|
||||
# unauthenticated episode-audio download.
|
||||
# Only 8502 is published on all interfaces, because it is the single upstream
|
||||
# the guard chain dials: notebook.franksdank.cloud → anubis → authentik →
|
||||
# http://:8017 → 192.168.0.143:8502 (dankclaw/Caddyfile), and this Caddy lives
|
||||
# in root's storage, so a published port is the only way it can reach a rootless
|
||||
# quadlet. The API needs no outward face at all — the image's own Next.js layer
|
||||
# rewrites /api/* to the FastAPI beside it, so one origin serves UI and API —
|
||||
# and the fleet's rule is that loopback-only is for services whose user path is
|
||||
# the guard chain, which this became on 2026-10-19; 5056 stays published on
|
||||
# loopback because the deployer's health gate dials it there. That is also why
|
||||
# API_URL in scripts/open-notebook.env names the vhost rather than a port: left
|
||||
# to itself the frontend auto-detects `<host-you-opened>:5055`, and the UI loads
|
||||
# while every call fails. OPEN_NOTEBOOK_PASSWORD is not made redundant by the
|
||||
# chain: it is what keeps the API honest for everything else on this host, and
|
||||
# with it unset the API answers anyone, including the unauthenticated
|
||||
# episode-audio download.
|
||||
#
|
||||
# VOLUMES. `/app/data` is a named volume: uploads, the LangGraph SQLite
|
||||
# checkpoints and the extraction caches the image keeps under it
|
||||
@@ -81,7 +85,7 @@ Pull=never
|
||||
Network=dankclaw_default
|
||||
Network=open-notebook.network
|
||||
PublishPort=0.0.0.0:8502:8502
|
||||
PublishPort=0.0.0.0:5056:5055
|
||||
PublishPort=127.0.0.1:5056:5055
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||
Volume=open-notebook-data:/app/data
|
||||
|
||||
@@ -45,3 +45,4 @@
|
||||
192.168.0.143 pihole.franksdank.cloud
|
||||
192.168.0.143 seerr.franksdank.cloud
|
||||
192.168.0.143 marreta.franksdank.cloud
|
||||
192.168.0.143 notebook.franksdank.cloud
|
||||
|
||||
@@ -19,11 +19,14 @@
|
||||
# is recorded in the parent tree, so cloning the parent commit recursively is
|
||||
# reproducible without a second pin.
|
||||
#
|
||||
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim:
|
||||
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim — with the Vulkan door
|
||||
# open since 2026-10-19. That configure was run in this exact base image and
|
||||
# answers `Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version
|
||||
# "1.3.275") found components: glslc` then `Including Vulkan backend`:
|
||||
#
|
||||
# cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
|
||||
# -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
|
||||
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF \
|
||||
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON \
|
||||
# -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
|
||||
# && cmake --build build -j 8
|
||||
#
|
||||
@@ -42,8 +45,12 @@
|
||||
# defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
|
||||
# host — correct while build host == run host, wrong if anyone ever builds
|
||||
# this image elsewhere, which is why the fleet builds it on Dankclaw.
|
||||
# CUDA/HIP/VULKAN/METAL/SYCL/RPC all OFF: this is a CPU endpoint (the trial
|
||||
# ran on CPU; the box's GPUs belong to the text endpoints).
|
||||
# CUDA/HIP/METAL/SYCL/RPC stay OFF, VULKAN is ON: the endpoint computes on the
|
||||
# gfx1151 iGPU (decision and co-tenancy gates:
|
||||
# report/tts-npu-spike-verdict-2026-10-19.md, shape in [[Frank TTS Engine]]).
|
||||
# Nothing else is available to this image: no ROCm userland, no NVIDIA
|
||||
# userland, no macOS, no oneAPI, and RPC would add a second process to
|
||||
# supervise on a box that already has one engine per card.
|
||||
#
|
||||
# BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
|
||||
# tts-server links libggml*.so rather than swallowing them; the runtime stage
|
||||
@@ -67,9 +74,28 @@ ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02f
|
||||
# ------------------------------------------------------------------ build
|
||||
FROM ${BASE_IMAGE} AS build
|
||||
|
||||
# The three Vulkan build deps are not optional and not interchangeable:
|
||||
# libvulkan-dev vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake
|
||||
# configs, i.e. what find_package(Vulkan) resolves to.
|
||||
# glslc ggml compiles its shaders with glslc, not glslangValidator.
|
||||
# Ubuntu's glslang-tools ships glslangValidator and NOT glslc,
|
||||
# so without this package `find_package(Vulkan COMPONENTS
|
||||
# glslc REQUIRED)` fails and there is no Vulkan backend.
|
||||
# The archive's glslc is old (2023.8): it compiles
|
||||
# GL_KHR_cooperative_matrix — the path ggml's quantized
|
||||
# matmul needs — but reports cooperative_matrix2,
|
||||
# integer_dot_product, bfloat16 and the micro-float
|
||||
# extensions as unsupported, so those shader families are
|
||||
# compiled out. If a Vulkan prefill ever reads as
|
||||
# inexplicably slow, suspect this compiler before the device;
|
||||
# the fix is a pinned glslang build passed as
|
||||
# -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
|
||||
# spirv-headers find_package(SPIRV-Headers CONFIG REQUIRED); the distro
|
||||
# package ships /usr/share/cmake/SPIRV-Headers/.
|
||||
RUN apt-get update -qq \
|
||||
&& apt-get install -y -qq --no-install-recommends \
|
||||
ca-certificates cmake g++ git make \
|
||||
libvulkan-dev glslc spirv-headers \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Checked out detached at a full SHA, then the recorded submodule. No branch
|
||||
@@ -90,7 +116,7 @@ RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
|
||||
-DGGML_BLAS=OFF \
|
||||
-DGGML_CUDA=OFF \
|
||||
-DGGML_HIP=OFF \
|
||||
-DGGML_VULKAN=OFF \
|
||||
-DGGML_VULKAN=ON \
|
||||
-DGGML_METAL=OFF \
|
||||
-DGGML_SYCL=OFF \
|
||||
-DGGML_RPC=OFF \
|
||||
@@ -102,9 +128,21 @@ FROM ${BASE_IMAGE}
|
||||
# libgomp1: the OpenMP runtime the build above links against. curl: the unit's
|
||||
# HealthCmd. ca-certificates: so that curl can speak https if anyone ever
|
||||
# points a probe at a URL with a scheme this host terminates elsewhere.
|
||||
#
|
||||
# libvulkan1 + mesa-vulkan-drivers are what make `--device Vulkan0` mean
|
||||
# anything at runtime: libvulkan1 is the loader libggml-vulkan.so dlopens, and
|
||||
# mesa-vulkan-drivers is the RADV ICD that claims the gfx1151. Loader without an
|
||||
# ICD enumerates ZERO devices, and tts-server then dies in device lookup AFTER
|
||||
# both GGUFs have been read — the journal reads as a model fault, not a missing
|
||||
# driver. Enumerated from this base on 2026-10-19 with `--device /dev/dri
|
||||
# --group-add keep-groups` plus the host's ComfyUI udev rule: "Radeon 8060S
|
||||
# Graphics (RADV GFX1151)", API 1.4.318 (the heap figure is owned by
|
||||
# [[Frank TTS Engine]]; it is the GTT-backed aperture, not the ~2 GiB DRM
|
||||
# carve-out). No NVIDIA ICD is installed on purpose: one Vulkan device in the
|
||||
# enumeration is what makes the NAME Vulkan0 safe to write down.
|
||||
RUN apt-get update -qq \
|
||||
&& apt-get install -y -qq --no-install-recommends \
|
||||
ca-certificates curl libgomp1 \
|
||||
ca-certificates curl libgomp1 libvulkan1 mesa-vulkan-drivers \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
|
||||
|
||||
@@ -40,12 +40,28 @@
|
||||
# speaks to the server from inside its own namespace, which is where the address
|
||||
# is valid.
|
||||
#
|
||||
# MEMORY / NO GPU. Peak RSS measured 5.7 GiB with both GGUFs resident
|
||||
# (2026-10-18 trial); no memory limit is set because the box has the room and a
|
||||
# limit would convert a busy moment into an OOM kill mid-utterance. No
|
||||
# /dev/kfd, no /dev/dri, no keep-groups: every GPU backend is compiled out of
|
||||
# the image, the endpoint is CPU-only by decision, and the GPUs on this host
|
||||
# belong to the text endpoints.
|
||||
# MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs:
|
||||
# the Vulkan backend copies both tensors into device memory at load, so they
|
||||
# stop counting in the process's RSS (measured after the 2026-10-19 cutover —
|
||||
# `podman stats` / `ps -o rss`). No memory limit is set: the box has the room
|
||||
# and a limit would convert a busy moment into an OOM kill mid-utterance.
|
||||
#
|
||||
# `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan
|
||||
# ICD enumerates nothing and the server dies on device lookup after it has
|
||||
# already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the
|
||||
# thing that works here: the units podman spawns carry no supplementary groups
|
||||
# at all, and this host grants render access a different way — the udev rule
|
||||
# that makes the render nodes world-writable (same rule ComfyUI rides, see
|
||||
# dankclaw/comfyui). Measured: `id` inside this container lists only the
|
||||
# container's own group, and `tts-server` still holds renderD129.
|
||||
# The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and
|
||||
# renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a
|
||||
# re-probe can invalidate, while the directory hands the container every render
|
||||
# node and the driver picks by name.
|
||||
#
|
||||
# /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
|
||||
# the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
|
||||
# container a second way onto the GPU is surface for nothing.
|
||||
#
|
||||
# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
|
||||
# becomes known to the engine only through POST /v1/audio/voices, and the
|
||||
@@ -98,6 +114,9 @@ Network=frank-tts-engine.network
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
|
||||
Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
|
||||
Volume=/home/dank/voices:/voices:ro
|
||||
# Render access for the iGPU (see MEMORY / THE RENDER NODE above).
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
HealthCmd=curl -fsS http://127.0.0.1:8880/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
# supervision, no retries, no logging of its own. PID 1 stays the server, so
|
||||
# `podman stop` and the unit's Restart=always drive it directly.
|
||||
#
|
||||
# THE FLAGS THAT ARE NOT HERE, and must stay absent:
|
||||
# THE FLAG THAT MUST STAY ABSENT:
|
||||
#
|
||||
# --threads / -t does not exist in this build. src/backend.h computes the
|
||||
# CPU thread count as std::thread::hardware_concurrency()/2
|
||||
@@ -18,13 +18,19 @@
|
||||
# placement — pinning is deliberately not done anywhere in
|
||||
# this stack. See the env file header for the same warning
|
||||
# aimed at the operator.
|
||||
# --device left at its default. Every GPU backend is compiled out of
|
||||
# the image, so the default device pick can only land on
|
||||
# CPU; naming a device would add a way to be wrong.
|
||||
#
|
||||
# Everything defaults to the shape the 2026-10-18 trial measured, so an empty
|
||||
# environment still boots the decided configuration: 1.7 B talker Q4_K_M with
|
||||
# the Q8_0 tokenizer, one request at a time, on the private bridge.
|
||||
# --device IS here, but conditionally: it reaches the binary only when
|
||||
# QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
|
||||
# takes its own default device, which is CPU because the CPU backend is always
|
||||
# built. One line in scripts/frank-tts-engine.env moves the endpoint between the
|
||||
# two shapes, with no rebuild. What must never reach argv is a device NAME the
|
||||
# loader does not enumerate: that is not refused by the argument parser, it dies
|
||||
# during device lookup with the loader's own error, so the journal reads as a
|
||||
# load that failed rather than a configuration refusal.
|
||||
#
|
||||
# Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
|
||||
# the Q8_0 tokenizer, one request at a time, on the private bridge, on the
|
||||
# gfx1151 iGPU. An empty environment still boots that shape.
|
||||
set -euo pipefail
|
||||
|
||||
# The binary path. Overridable for one reason: the bats seam runs a stub
|
||||
@@ -60,13 +66,33 @@ set -euo pipefail
|
||||
: "${QWENTTS_LANG:=auto}"
|
||||
|
||||
# One request at a time. The server-side queue is a serialised FIFO at this
|
||||
# setting, which is what keeps the measured per-voice timings true; batching
|
||||
# on a CPU build splits the same cores between two syntheses and the warm RTF
|
||||
# margin (0.55-0.67 against real time 1.0) disappears. The engine's own
|
||||
# default is also 1; it is written here because it is a decision, not an
|
||||
# accident.
|
||||
# setting, which is what keeps the per-voice timings predictable. A second
|
||||
# concurrent synthesis shares the device with the first — and on this box that
|
||||
# device is also carrying frank-dragoon — so it buys no throughput and it
|
||||
# doubles the wait for the caller behind it. The engine's own default is also 1;
|
||||
# written here because it is a decision, not an accident.
|
||||
: "${QWENTTS_MAX_BATCH:=1}"
|
||||
|
||||
# The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
|
||||
# the gfx1151 iGPU on this host. An index would follow whatever order the
|
||||
# loader's device list returns; a name says which device this endpoint means.
|
||||
# The container sees exactly one Vulkan device because the image carries the
|
||||
# Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
|
||||
#
|
||||
# EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
|
||||
# the binary picks its own default, and the CPU backend is always built — which
|
||||
# is why moving this endpoint between the iGPU and the CPU is one line in
|
||||
# scripts/frank-tts-engine.env and not a rebuild.
|
||||
#
|
||||
# That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
|
||||
# fires on a set-but-empty variable as well as an unset one, so the rollback line
|
||||
# `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
|
||||
# on the shared GPU with no way out short of a rebuild. Only UNSET takes the
|
||||
# default.
|
||||
if [[ ! -v QWENTTS_DEVICE ]]; then
|
||||
QWENTTS_DEVICE=Vulkan0
|
||||
fi
|
||||
|
||||
# Fail loudly BEFORE binding, so the journal says which artifact is missing
|
||||
# instead of the unit reporting a container that starts and dies. This is the
|
||||
# self-healing path: Restart=always re-runs it every RestartSec, and the same
|
||||
@@ -79,11 +105,20 @@ for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
|
||||
fi
|
||||
done
|
||||
|
||||
exec "$QWENTTS_BIN" \
|
||||
--model "$QWENTTS_MODEL" \
|
||||
--codec "$QWENTTS_CODEC" \
|
||||
--alias "$QWENTTS_ALIAS" \
|
||||
--host "$QWENTTS_HOST" \
|
||||
--port "$QWENTTS_PORT" \
|
||||
--lang "$QWENTTS_LANG" \
|
||||
ARGS=(
|
||||
--model "$QWENTTS_MODEL"
|
||||
--codec "$QWENTTS_CODEC"
|
||||
--alias "$QWENTTS_ALIAS"
|
||||
--host "$QWENTTS_HOST"
|
||||
--port "$QWENTTS_PORT"
|
||||
--lang "$QWENTTS_LANG"
|
||||
--max-batch "$QWENTTS_MAX_BATCH"
|
||||
)
|
||||
|
||||
# Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
|
||||
# because "the rollback shape" is argv identical to the pre-iGPU endpoint.
|
||||
if [[ -n "$QWENTTS_DEVICE" ]]; then
|
||||
ARGS+=(--device "$QWENTTS_DEVICE")
|
||||
fi
|
||||
|
||||
exec "$QWENTTS_BIN" "${ARGS[@]}"
|
||||
|
||||
@@ -0,0 +1,319 @@
|
||||
# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
|
||||
|
||||
Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
|
||||
Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
|
||||
2026-10, and whether any of them can serve a speech-synthesis model.
|
||||
|
||||
> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
|
||||
> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
|
||||
> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
|
||||
> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
|
||||
|
||||
Convention: vendor claims carry URL + the version/date they hold for.
|
||||
**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
|
||||
vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
|
||||
|
||||
## 0. Answers to the three questions Main ranked
|
||||
|
||||
**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
|
||||
host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
|
||||
lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
|
||||
*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
|
||||
release note "New TTS model support: neutts-air, neutts-nano" —
|
||||
<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
|
||||
published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
|
||||
2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
|
||||
Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
|
||||
**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
|
||||
`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
|
||||
`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
|
||||
reference-audio encoder**. [INFERENCE from HF file lists]
|
||||
|
||||
**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
|
||||
**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
|
||||
|
||||
**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
|
||||
ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
|
||||
generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
|
||||
lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
|
||||
|
||||
## 1. Landscape: every way to reach the XDNA2 array on Linux
|
||||
|
||||
| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
|
||||
|---|------|---------|------------------------------|------------------|
|
||||
| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
|
||||
| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
|
||||
| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
|
||||
| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
|
||||
| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
|
||||
| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
|
||||
| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
|
||||
| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
|
||||
|
||||
Sources: §2–§6 below.
|
||||
|
||||
## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
|
||||
|
||||
* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
|
||||
v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
|
||||
Kraken, and Gorgon Point)."
|
||||
<https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
|
||||
* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
|
||||
generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
|
||||
`amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
|
||||
`amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
|
||||
run` goes through **XRT** and fails with `No such device with index '0'` unless
|
||||
`xrt-smi examine` lists the NPU; `memlock` must be raised.
|
||||
<https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
|
||||
2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
|
||||
distro package on the host **[host]**.
|
||||
* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
|
||||
auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
|
||||
`v1/audio/transcriptions`, `v1/embeddings`. **There is no
|
||||
`/v1/audio/speech`.**
|
||||
<https://fastflowlm.com/docs/instructions/server/openapi/>
|
||||
* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
|
||||
Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
|
||||
**Whisper — card states "Type: Speech-to-Text (ASR)"**
|
||||
(<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
|
||||
Hy-MT2, MiniCPM. No speech-synthesis entry.
|
||||
* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
|
||||
("Internet access to HuggingFace is required to download the optimized model
|
||||
kernels"). The only in-tree engine build is Gemma-4E
|
||||
(`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
|
||||
→ **A new TTS architecture cannot be added to `flm` by us.**
|
||||
|
||||
## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
|
||||
|
||||
* **Install channel (important correction to the pypi.amd.com evidence).** The pip
|
||||
index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
|
||||
win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
|
||||
single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
|
||||
as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
|
||||
account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
|
||||
→ HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
|
||||
(**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
|
||||
EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
|
||||
serves it directly. Verified by me with curl 2026-10-19. I did not finish
|
||||
downloading 7.8 GiB, so the tarball's *contents* list is unverified —
|
||||
`[INFERENCE]` only where noted.)
|
||||
* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
|
||||
`https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
|
||||
contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
|
||||
`xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
|
||||
download + unpack). **The plugin deb ships firmware for
|
||||
`/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
|
||||
NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
|
||||
`libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
|
||||
source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
|
||||
driver + XRT shim + firmware only.
|
||||
* **Our device is officially STX.** AMD's APU table
|
||||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
|
||||
`0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
|
||||
= PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
|
||||
column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
|
||||
* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
|
||||
2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
|
||||
"amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
|
||||
has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
|
||||
reports the device as **NPU Strix Halo**.
|
||||
* Linux install recipe (1.8.0 docs
|
||||
<https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
|
||||
**Python 3.12** (`apt install python3.12 python3.12-venv`),
|
||||
`libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
|
||||
`LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
|
||||
`quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
|
||||
Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
|
||||
flows. "Linux currently supports NPU only flow" (no iGPU EP).
|
||||
Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
|
||||
must be built with a 3.12 interpreter and the packages are 24.04-built
|
||||
`[INFERENCE]`.
|
||||
* LLM flow on Linux
|
||||
(<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
|
||||
ready-to-run models from HF, `deployment/lib`,
|
||||
`RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
|
||||
run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
|
||||
**"Model generation is not supported in current release"** (repeated in
|
||||
relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
|
||||
Linux"); needs `memlock unlimited`.
|
||||
Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
|
||||
(Phi-3.5-mini, 4K model).
|
||||
* Third-party confirmation of the OS split: Lemonade's backend matrix
|
||||
(<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
|
||||
fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
|
||||
`ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
|
||||
text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
|
||||
**no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
|
||||
Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
|
||||
|
||||
## 4. Can we compile our own graph on Linux? (the no-recipe path)
|
||||
|
||||
Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
|
||||
ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
|
||||
caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
|
||||
"precompile the EP context model using Python and deploy it using a C++ program"
|
||||
is explicitly supported); EP auto-partitions unsupported nodes to CPU;
|
||||
BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
|
||||
BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
|
||||
"general asymmetric quantization … to enable third-party quantized models";
|
||||
**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
|
||||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
|
||||
cache dirs are not portable across EP or NPU-driver versions.
|
||||
|
||||
Operator coverage is the hard gate
|
||||
(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
|
||||
Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
|
||||
|
||||
* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
|
||||
(absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
|
||||
`CumSum` (BF16/XINT8), `STFT` (XINT8 only).
|
||||
* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
|
||||
table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
|
||||
XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
|
||||
the **LLM/OGA flow** (whose op set is exactly
|
||||
`SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
|
||||
(W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
|
||||
`Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
|
||||
* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
|
||||
`NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
|
||||
`RMSNormalization` → graphs that emit them must be rewritten before they
|
||||
partition onto the NPU. [INFERENCE from absence in the table.]
|
||||
* Quantization is a separate tool, not in the driver package: **AMD Quark**
|
||||
(`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
|
||||
softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
|
||||
on pypi.org under the obvious names **[host probe]** and ships from AMD's
|
||||
gated/pip-private channel.
|
||||
* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
|
||||
OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
|
||||
speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
|
||||
Windows box) in the loop.
|
||||
* Practical shape cost: the neutts artifacts carry **one compiled variant per
|
||||
padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
|
||||
prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
|
||||
artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
|
||||
|
||||
## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
|
||||
|
||||
NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
|
||||
clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
|
||||
codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
|
||||
`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
|
||||
require **pre-encoded references** — these ship with the repo for all bundled
|
||||
voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
|
||||
active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
|
||||
(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
|
||||
codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
|
||||
(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
|
||||
2026-08-26, `qwen2`, GGUF + safetensors).
|
||||
|
||||
Consequences for the fleet spike:
|
||||
|
||||
1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
|
||||
work either way**. Our `reference.wav` voices survive only if we run the
|
||||
NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
|
||||
once per voice and then feed pre-encoded codes. That is a one-time offline
|
||||
step per voice; after that the compiled graph takes codes/prompt prefix as
|
||||
ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
|
||||
speaker prefix. `[INFERENCE]`
|
||||
2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
|
||||
Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
|
||||
roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
|
||||
`[INFERENCE — different model; must be measured.]`
|
||||
3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
|
||||
NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
|
||||
3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
|
||||
4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
|
||||
only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
|
||||
(<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
|
||||
tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
|
||||
`Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
|
||||
— no TTS directory. Serving TTS therefore means writing our own OGA host
|
||||
(or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
|
||||
|
||||
## 6. Two graphs on one NPU, and the iGPU hazard
|
||||
|
||||
* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
|
||||
(in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
|
||||
npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
|
||||
context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
|
||||
(`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
|
||||
`fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
|
||||
So halogen's armed models and a TTS graph can hold separate contexts
|
||||
simultaneously; they contend for columns and OPC share, they are not mutually
|
||||
exclusive.
|
||||
* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
|
||||
contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
|
||||
resource distribution across applications"; 1.3 "Idle detection and suspension
|
||||
of contexts", "Rebalance feature for AIE hardware resource optimization";
|
||||
1.0 known issue "Support for multiple applications is limited to up to eight"
|
||||
(all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
|
||||
* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
|
||||
XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
|
||||
1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
|
||||
— the vendor README warns that a stale/mismatched `npu.sbin` causes
|
||||
`ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
|
||||
than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
|
||||
`EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
|
||||
fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
|
||||
under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
|
||||
the whole host.
|
||||
* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
|
||||
vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
|
||||
optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
|
||||
system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
|
||||
knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
|
||||
(<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
|
||||
<https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
|
||||
The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
|
||||
fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
|
||||
|
||||
## 7. Paths I checked and ruled out (so nobody re-walks them)
|
||||
|
||||
* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
|
||||
`ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
|
||||
generation. Not usable on Dankclaw.
|
||||
* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
|
||||
AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
|
||||
(<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
|
||||
EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
|
||||
deployment dir, per llm_linux).
|
||||
* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
|
||||
(`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
|
||||
NPU; migration means a different runtime, not a flag.
|
||||
* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
|
||||
by install cost, and it is not a door for TTS.
|
||||
* **IRON / mlir-aie** is the only open toolchain that documents running on the
|
||||
Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
|
||||
AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
|
||||
Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
|
||||
`ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
|
||||
required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
|
||||
fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
|
||||
optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
|
||||
hand-written per-op tile programming, not a model compiler: writing a TTS model
|
||||
here is a multi-month engineering project, not a migration.
|
||||
* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
|
||||
|
||||
## 8. Recommendation implied by the above (for Main's spike)
|
||||
|
||||
Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
|
||||
+ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
|
||||
distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
|
||||
"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
|
||||
each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
|
||||
is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
|
||||
the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
|
||||
serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
|
||||
|
||||
## 9. Loose ends (not verified; I stopped on Main's stand-down)
|
||||
|
||||
* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
|
||||
the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
|
||||
and which Python wheels it carries. My statement that non-LLM compilation works
|
||||
on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
|
||||
… NPU"), not on the package contents. Download was killed mid-flight.
|
||||
* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
|
||||
runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
|
||||
documented anywhere in 1.8.0 docs).
|
||||
* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
|
||||
from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).
|
||||
@@ -0,0 +1,438 @@
|
||||
# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability
|
||||
|
||||
Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
|
||||
portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
|
||||
runtimes exist on Dankclaw).
|
||||
|
||||
Evidence date: 2026-10-19. Everything below is grounded in:
|
||||
|
||||
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
|
||||
(2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
|
||||
`ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
|
||||
- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
|
||||
byte-for-byte from the live host.
|
||||
- Upstream Qwen3-TTS model cards / release material.
|
||||
|
||||
Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
|
||||
night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
|
||||
`libggml-cpu.so.0.27.0` and nothing else, and our own
|
||||
`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
|
||||
the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
|
||||
exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1. What the payload is
|
||||
|
||||
Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
|
||||
owned `dank:dank`, mtimes 2026-10-18 18:31), both from
|
||||
`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):
|
||||
|
||||
| file | bytes | GGUF | tensors | arch string | file_type |
|
||||
|---|---|---|---|---|---|
|
||||
| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
|
||||
| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |
|
||||
|
||||
`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
|
||||
to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
|
||||
files is unaccounted for.
|
||||
|
||||
### 1.1 Talker file (480 tensors)
|
||||
|
||||
Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
|
||||
(qwentts.cpp @ pinned commit) and confirmed by the tensor census:
|
||||
|
||||
| prefix | tensors | bytes | what it is |
|
||||
|---|---|---|---|
|
||||
| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
|
||||
| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
|
||||
| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |
|
||||
|
||||
Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
|
||||
`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.
|
||||
|
||||
Which tensors carry which quant is the load-bearing portability fact:
|
||||
|
||||
- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
|
||||
[2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
|
||||
These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
|
||||
embedding tables rather than to an `lm_head`.
|
||||
- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
|
||||
Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
|
||||
`[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
|
||||
`[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
|
||||
GEMM shapes, nothing exotic.
|
||||
- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
|
||||
block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
|
||||
`quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
|
||||
the `text_proj` biases.
|
||||
|
||||
Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
|
||||
`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
|
||||
`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
|
||||
the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
|
||||
think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
|
||||
portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
|
||||
special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
|
||||
the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
|
||||
repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
|
||||
**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
|
||||
eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.
|
||||
|
||||
Two live-file details that contradict the upstream doc text (worth knowing when reading
|
||||
`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
|
||||
`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
|
||||
(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
|
||||
build shipped here has 480 because it carries `spk_enc.*`.
|
||||
|
||||
### 1.2 Tokenizer/codec file (398 tensors)
|
||||
|
||||
One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
|
||||
entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
|
||||
`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.
|
||||
|
||||
Composition (metadata + tensor census, both files agree):
|
||||
|
||||
- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
|
||||
conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
|
||||
k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
|
||||
LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
|
||||
argmin loop. Metadata records `encoder.num_quantizers=32` with
|
||||
`encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
|
||||
the upstream config.
|
||||
- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
|
||||
`output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
|
||||
head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
|
||||
1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
|
||||
transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
|
||||
(`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
|
||||
are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
|
||||
→ snake_post → conv_post k=7 → 96→1.
|
||||
- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
|
||||
deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
|
||||
wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
|
||||
quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
|
||||
`quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
|
||||
(7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
|
||||
- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
|
||||
(Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
|
||||
NPU question: every component is convolution, attention, or embedding lookup.
|
||||
|
||||
### 1.3 Where the payload came from, and what the official weights look like
|
||||
|
||||
Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):
|
||||
|
||||
| repo | last modified | license | weights |
|
||||
|---|---|---|---|
|
||||
| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
|
||||
| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
|
||||
| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
|
||||
| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
|
||||
| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |
|
||||
|
||||
So the GGUF is **not** the only public form: the safetensors are the canonical form
|
||||
(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
|
||||
them produced by the same author who wrote the runtime (`Serveurperso`), via
|
||||
`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
|
||||
Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
|
||||
(that count includes the embedded codec), the codec alone is 170.6 M params in F32.
|
||||
|
||||
**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
|
||||
block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
|
||||
ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
|
||||
quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
|
||||
1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
|
||||
payload to a non-ggml stack therefore means starting from the safetensors again, not
|
||||
converting the GGUF.
|
||||
|
||||
**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
|
||||
converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
|
||||
non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
|
||||
fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
|
||||
constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
|
||||
convs are not, at least not without re-measuring voice quality.
|
||||
|
||||
## 2. What the runtime is
|
||||
|
||||
`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
|
||||
application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
|
||||
`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
|
||||
range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
|
||||
build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.
|
||||
|
||||
Five graph families, and their shapes are the whole portability story:
|
||||
|
||||
| graph | batch/time axis | notes |
|
||||
|---|---|---|
|
||||
| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
|
||||
| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
|
||||
| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
|
||||
| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
|
||||
| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |
|
||||
|
||||
Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
|
||||
audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
|
||||
fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
|
||||
batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
|
||||
chunked (buffered) codec decode.
|
||||
|
||||
Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
|
||||
`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
|
||||
`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
|
||||
`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
|
||||
`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.
|
||||
|
||||
**No custom ggml op is required today.** The repo doc still says the fork "provides the two
|
||||
custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
|
||||
pinned commit. Verified 2026-10-19:
|
||||
|
||||
- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
|
||||
(`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
|
||||
`include/ggml.h` is byte-identical to upstream master (`diff` empty);
|
||||
- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
|
||||
(mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
|
||||
builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
|
||||
and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
|
||||
- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
|
||||
the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
|
||||
(`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).
|
||||
|
||||
Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
|
||||
payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.
|
||||
|
||||
Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
|
||||
`CPU…`, default best available), resolved through the ggml backend registry; the deployed
|
||||
container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
|
||||
backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
|
||||
The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
|
||||
**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
|
||||
(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).
|
||||
|
||||
Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
|
||||
empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
|
||||
`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
|
||||
(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
|
||||
mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
|
||||
(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
|
||||
(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.
|
||||
|
||||
Parity expectations for any port: the upstream harness matches the PyTorch reference per
|
||||
stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
|
||||
reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
|
||||
one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
|
||||
accepted by listening + per-stage cosine, never by waveform diff against today's audio.
|
||||
|
||||
---
|
||||
|
||||
## 3. The three answers Main asked for
|
||||
|
||||
### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter
|
||||
|
||||
There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
|
||||
container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
|
||||
GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
|
||||
op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
|
||||
official Apache-2.0 safetensors are upstream, and **four independent ports of the full
|
||||
pipeline already exist**:
|
||||
|
||||
| port | form | what it proves |
|
||||
|---|---|---|
|
||||
| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
|
||||
| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
|
||||
| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
|
||||
| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |
|
||||
|
||||
What the ports had to concede, and what any port inherits:
|
||||
|
||||
- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
|
||||
support this generation structure yet", so their first shape was **three graphs + a
|
||||
host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
|
||||
- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
|
||||
loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
|
||||
CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
|
||||
- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
|
||||
0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
|
||||
**warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
|
||||
An NPU port is not "faster by default" for this workload; it has to beat that.
|
||||
- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
|
||||
in fp16/fp32 (the LiteRT split does exactly this).
|
||||
|
||||
So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
|
||||
from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
|
||||
static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
|
||||
precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
|
||||
fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.
|
||||
|
||||
### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS
|
||||
|
||||
Checked against the trees today (2026-10-19), because this was the crux:
|
||||
|
||||
- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
|
||||
`xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
|
||||
HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
|
||||
OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
|
||||
matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
|
||||
opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
|
||||
2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
|
||||
- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
|
||||
`GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
|
||||
GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
|
||||
(`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
|
||||
token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
|
||||
(HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
|
||||
new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
|
||||
unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
|
||||
2,048-wide **lose** (0.77× at width 1,536).
|
||||
- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
|
||||
explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
|
||||
amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
|
||||
mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
|
||||
int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
|
||||
different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
|
||||
12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
|
||||
use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
|
||||
conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
|
||||
AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
|
||||
picture of what XDNA2 graph execution costs.
|
||||
- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
|
||||
LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
|
||||
`hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.
|
||||
|
||||
Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
|
||||
by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
|
||||
sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
|
||||
covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
|
||||
elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
|
||||
this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
|
||||
graph on the NPU", which is exactly the NeuTTS-Air spike.
|
||||
|
||||
### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP
|
||||
|
||||
- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
|
||||
(Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
|
||||
runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
|
||||
`GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
|
||||
OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
|
||||
`--device Vulkan0` on the same command line the unit runs today.
|
||||
- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
|
||||
whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
|
||||
`swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
|
||||
`CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
|
||||
WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
|
||||
transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
|
||||
- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
|
||||
runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
|
||||
nothing in the tree is NVIDIA-specific.
|
||||
- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
|
||||
that the hardware/driver combination is viable: this very host already runs a Vulkan
|
||||
endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
|
||||
implementation is officially supported on Radeon Software for Linux
|
||||
(AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
|
||||
a build-and-measure spike, not an unsupported port.
|
||||
- The real constraint is the one already in the fleet docs: the iGPU belongs to another
|
||||
endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
|
||||
with the text endpoints for the same silicon; the NPU route's only genuine advantage is
|
||||
that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
|
||||
is busy … no GPU contention").
|
||||
|
||||
---
|
||||
|
||||
## 4. What any replacement must reproduce, and what the voices cost
|
||||
|
||||
Endpoint contract in production today (from the live unit/env, not the doc):
|
||||
`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
|
||||
--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
|
||||
port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
|
||||
`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
|
||||
`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
|
||||
`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
|
||||
Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
|
||||
choice out of every consumer's config.
|
||||
|
||||
Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
|
||||
(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
|
||||
`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
|
||||
silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
|
||||
artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
|
||||
`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
|
||||
re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
|
||||
registration time.
|
||||
|
||||
That is the portable part, and it is the same shape AMD's NeuTTS flow wants
|
||||
(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
|
||||
*the WAV plus its transcript*, processed at request/registration time by the model's own
|
||||
speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
|
||||
inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
|
||||
`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
|
||||
replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
|
||||
re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
|
||||
that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
|
||||
(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
|
||||
(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
|
||||
ships a 1024-dim one).
|
||||
|
||||
One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
|
||||
engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
|
||||
interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
|
||||
regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.
|
||||
|
||||
## 5. Sources
|
||||
|
||||
Runtime/payload (all read locally at pinned versions, 2026-10-19):
|
||||
|
||||
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
|
||||
(commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
|
||||
(860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
|
||||
- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
|
||||
delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
|
||||
https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
|
||||
`include/ggml.h` identical to upstream
|
||||
https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
|
||||
- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
|
||||
(1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
|
||||
the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
|
||||
- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
|
||||
`scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.
|
||||
|
||||
Model provenance:
|
||||
|
||||
- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
|
||||
VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
|
||||
2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
|
||||
`speech_tokenizer/`) and 170,557,441 (F32 codec).
|
||||
- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
|
||||
`generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
|
||||
`create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
|
||||
flash-attn-2, Python 3.12 recommended).
|
||||
- `https://github.com/QwenLM/Qwen3-TTS`.
|
||||
|
||||
Portability precedents:
|
||||
|
||||
- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
|
||||
`https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
|
||||
`https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
|
||||
`https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
|
||||
`https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
|
||||
`legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).
|
||||
|
||||
AMD NPU / ggml backend status:
|
||||
|
||||
- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
|
||||
XDNA/AIE/Vitis backend, 2026-10-19) ·
|
||||
`https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
|
||||
`https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
|
||||
- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
|
||||
measured 2026-10-05/06 on HX 370) ·
|
||||
`https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
|
||||
phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
|
||||
- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
|
||||
only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
|
||||
**no TTS example anywhere in the official Ryzen AI Software tree**.
|
||||
- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
|
||||
models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
|
||||
- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
|
||||
(Mesa Vulkan driver officially supported).
|
||||
|
||||
Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
|
||||
portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
|
||||
which is flagged there.
|
||||
@@ -0,0 +1,191 @@
|
||||
# TTS on the Strix Halo NPU — spike verdict (2026-10-19)
|
||||
|
||||
Question: **how can the fleet's TTS engine run on hardware rather than CPU?** Answer in
|
||||
one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is
|
||||
the gfx1151 iGPU behind a build flag we already own.
|
||||
|
||||
Spike evidence lives in `report/npu-tts-runtime-landscape-2026-10-19.md` (which runtimes can
|
||||
reach XDNA2 at all) and `report/qwen3-tts-payload-portability-2026-10-19.md` (what the
|
||||
payload is and where it can be moved). This page owns only the **measured spike result** and
|
||||
the **model-side contract** neither slice has. Throwaway work ran in containers under
|
||||
`/home/dank/spikes/npu-tts` on Dankclaw; nothing in the fleet was changed — `frank-tts` and
|
||||
`frank-tts-engine` still serve `narrator1|2|3` on CPU.
|
||||
|
||||
Legend: **[host]** live census 2026-10-19 · **[spike]** measured in the spike containers ·
|
||||
**[vendor]** primary vendor source, with the version it holds for.
|
||||
|
||||
## 1. Verdict
|
||||
|
||||
**NO-GO on the NPU, and the wall is a loader.** The XDNA2 device is healthy by AMD's own
|
||||
instruments; the runtime that owns this model family opens the device and then declines the
|
||||
graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a
|
||||
path that cannot be reproduced is not a foundation. **[spike]**
|
||||
|
||||
**Go on the iGPU, and the lever is in this repo.**
|
||||
`dankclaw/voiceservice/Containerfile.engine` builds `qwentts.cpp` (commit `5916bf7`) with
|
||||
`-DGGML_VULKAN=OFF` written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running
|
||||
container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-server` and
|
||||
nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
|
||||
the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
|
||||
world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
|
||||
gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
|
||||
the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
|
||||
budget while dragoon is loaded, so memory is not the constraint — which leaves the
|
||||
contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
|
||||
whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
|
||||
|
||||
Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
|
||||
only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
|
||||
dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan
|
||||
exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.
|
||||
|
||||
## 2. The two arms do not share a real-time bar
|
||||
|
||||
|Arm|Talker|Codec|Codes per second of audio|Bar|
|
||||
|---|---|---|---|---|
|
||||
|**Deployed** (`frank-tts-engine`)|`qwen-talker-1.7b-base-Q4_K_M.gguf`|`qwen-tokenizer-12hz-Q8_0.gguf`|**12.5**|~13 tok/s **[host]**|
|
||||
|**Spike** (`amd/neutts-air_rai_1.8.0_npu_4K`)|NuTTS-Air, 24-layer Qwen2|NeuCodec `24000 / hop 480`|**50.00**|~50 tok/s **[spike]**|
|
||||
|
||||
Measured from the staged NPU artifact — the contract any future arm must satisfy:
|
||||
|
||||
|Fact|Value|
|
||||
|---|---|
|
||||
Vocab|`genai_config.json` `vocab_size` **217,652**; base BPE 151,643; specials 151,644–151,670 **[spike]**|
|
||||
Speech tokens|**65,536** `<|speech_N|>` entries, ids **151,671–217,206** contiguous; the vendor template maps a codec index straight to its id, `"".join(f"<|speech_{i}|>")`. A 10,001-code assumption silently truncates the space — encoded references really do reach the 60k range. **[spike]**|
|
||||
Stop token|`<|SPEECH_GENERATION_END|>` = 151,670, but the export's `model.eos_token_id` is `[151645, 151643]` (the *chat turn* stop) and `search.eos_token_ids` is unset — pass 151,670 explicitly or speech ends after one token **[spike]**|
|
||||
Input ceiling|4096 tokens **total**, input + output **[vendor]**|
|
||||
Frame rate|50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) **[spike]**|
|
||||
Reference encode|`librosa.load(sr=16000, mono=True)` → `[1,1,T]` float → `codec.encode_code(audio_or_path=…)` (vendor `NeuTTS.encode_reference`); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. `neucodec` also needs `torchtune`; install both `--no-deps --target <dir>` so a second torch never shadows the venv's. **[spike]**|
|
||||
Pre-encoded voices|`/spike/logs/codes-narrator{1,2,3}.npy` — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s **[spike]**|
|
||||
|
||||
**Gap that any arm inherits:** the voice packs carry no transcript. `manifest.json` holds
|
||||
provenance and crop geometry (`crop.source_speech_run` 98.739–118.636 s of a 22,569 s
|
||||
audiobook) and no text, and Dankclaw has no ASR at all — `whisper`, `whisper.cpp`,
|
||||
`faster-whisper` all absent. NuTTS conditioning needs `ref_text`; do not invent it. **[host]**
|
||||
|
||||
## 3. Exoneration chain — why the blame is not the hardware
|
||||
|
||||
Every row is an instrument, not an inference.
|
||||
|
||||
|Probe|Result|
|
||||
|---|---|
|
||||
`xrt-smi examine`|`[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8`, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, `amdxdna 7.0.0-38-generic` **[spike]**|
|
||||
`xrt-smi validate`|**PASSED**, including `throughput` **[spike]**|
|
||||
`pyxrt.device(0)` in-container|constructs — XRT enumerates the device **[spike]**|
|
||||
`strace -f -e trace=openat` on the failing load|`genai_config.json` read → `model.onnx` opened → `libxrt_driver_xdna.so.2` loaded → `/dev/accel/accel0` opened `O_RDWR` → GEM ioctls succeed → **EP declines, before touching `dd_metastate_*` or `cache/txn_bins.zip`** **[spike]**|
|
||||
`dmesg`, `fuser /dev/accel/accel0`|silent since boot, no holder — the decline never reaches the kernel **[spike]**|
|
||||
Hardlinked mirror dirs of the model tree (`rundir*`)|retired, no change — the "metastate shadowing" theory is dead **[spike]**|
|
||||
Every logging lever in that build: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT`|**zero lines** — the decline path is silent **[spike]**|
|
||||
Vendor `quicktest`|`Session successfully initialized` — proves the XRT/driver/device path, **not** NPU residency: that build falls back to CPU silently. Prove residency with `/sys/class/accel/accel0/device/mem_usage` during generation. **[spike] [vendor]**|
|
||||
|
||||
No NPU token rate was ever measured, so none is quoted here. The single successful load
|
||||
(13:17, `[load] 0.84s`, finite logits, XRT banner at *generate* time) has not been
|
||||
reproduced by the byte-identical command since, with the model files pristine by inode and
|
||||
mtime. **[spike]**
|
||||
|
||||
## 4. Host gates the spike proved (so they are not re-litigated)
|
||||
|
||||
- **Rootless containers cannot read `/dev/accel/accel0`** (`root:render` 0660): the host
|
||||
gid never enters the userns map. `--group-add keep-groups`, `--userns=keep-id`,
|
||||
`--userns=keep-id:gid=990`, `--group-add render`, `--group-add gid:990` — all fail.
|
||||
Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet
|
||||
already does for `kfd`/`renderD*`. **[spike]**
|
||||
- **Never `dpkg -i` the vendor XRT deb** — it installs a DKMS `amdxdna` over the distro
|
||||
driver. `dpkg-deb -x` the debs and bind-mount the tree at `/opt/xilinx/xrt`; the plugin
|
||||
deb is what carries `libxrt_driver_xdna.so`. **[spike]**
|
||||
- In-tree `amdxdna 0.7.0-38` + `iommu=pt` is enough for context creation; a missing device
|
||||
node is *not* the modern failure mode. **[host]**
|
||||
- `xrt-smi` inside a container needs `libboost-filesystem1.83.0` +
|
||||
`libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`);
|
||||
boost is needed by `xrt-smi` only, never by the OGA/EP path. **[spike]**
|
||||
- Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census.
|
||||
A multi-GB NPU partition plus an AIE compile workspace does not fit beside
|
||||
`frank-dragoon`. Benign-looking `Failed to alloc dev bo memory, ret -28` is that
|
||||
pressure. **[host]**
|
||||
|
||||
## 5. Target shape: the TTS engine shares the gfx1151 with `frank-dragoon`
|
||||
|
||||
The deployment Frank wants is **both resident at once**, not a handover. Measured 2026-10-19
|
||||
(read-only, nothing loaded or unloaded):
|
||||
|
||||
|Measure|Value|Reading|
|
||||
|---|---|---|
|
||||
iGPU GTT in use|**18,523 MiB of 116,733 MiB**|≈96 GiB of aperture free; this is the real budget (fleet law) **[host]**|
|
||||
DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated — **not** the budget **[host]**|
|
||||
Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
|
||||
TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
|
||||
|
||||
**Two gates, in this order.**
|
||||
|
||||
1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
|
||||
carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
|
||||
`mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
|
||||
`Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
|
||||
|
||||
|Heap|size|budget with dragoon loaded|flags|
|
||||
|---|---|---|---|
|
||||
|0|38.67 GiB|32.58 GiB|host-visible, not device-local|
|
||||
|1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
|
||||
|
||||
The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
|
||||
a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
|
||||
of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
|
||||
and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
|
||||
Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
|
||||
help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
|
||||
ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
|
||||
device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
|
||||
3090 slot.
|
||||
2. **Contention gate — PASSED, measured 2026-10-19 after the cutover.** With a 16.9 s
|
||||
synthesis occupying the device, `frank-dragoon` answered a streamed 124-token request at
|
||||
**17 ms to first token and ~35 tok/s decode**, against 17 ms and ~33 tok/s for the same
|
||||
request with the card idle. Compute-unit sharing showed up as nothing. One request per
|
||||
arm, no ladder: these are the gate's evidence, not a throughput record, and the engine's
|
||||
own shape lives in [[Frank TTS Engine]].
|
||||
|
||||
**Change surface — as landed** (PR #34, `6248f02`; the CPU shape stays reachable as the
|
||||
rollback):
|
||||
|
||||
1. `Containerfile.engine` build stage: add **`libvulkan-dev glslc spirv-headers`**. The
|
||||
prediction above (`libvulkan-dev glslang-dev`) is wrong on two counts and the configure
|
||||
says so: ggml looks for `glslc`, which Ubuntu's `glslang-tools` does **not** ship (it
|
||||
ships `glslangValidator`), and `find_package(SPIRV-Headers CONFIG REQUIRED)` needs the
|
||||
distro's `spirv-headers`, which does carry the CMake config. `glslc` is its own source
|
||||
package on noble, at 2023.8 — old enough that only `GL_KHR_cooperative_matrix` compiles,
|
||||
so the coopmat2 / integer-dot / bf16 / micro-float shader families are built out. Suspect
|
||||
that compiler before the device if a Vulkan prefill ever reads as slow.
|
||||
2. `-DGGML_VULKAN=OFF` → `ON`: one word in the written-out flag list.
|
||||
3. Runtime stage: `mesa-vulkan-drivers` **and** `libvulkan1` — the ICD and its loader. An
|
||||
image with a loader and no ICD enumerates zero devices.
|
||||
4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — the
|
||||
DIRECTORY, never the node: on this hybrid box `renderD128` is the NVIDIA card and
|
||||
`renderD129` the amdgpu iGPU (`readlink -f /sys/class/drm/renderD129/device/driver` →
|
||||
`amdgpu`). On this host the udev rule already makes the nodes world-writable, so the
|
||||
group is belt-and-braces, not the gate.
|
||||
5. Engine argv: `--device Vulkan0`, reached through `QWENTTS_DEVICE` in
|
||||
`scripts/frank-tts-engine.env`. **An empty value passes no flag at all** — that is the
|
||||
CPU rollback, and it is why the entrypoint's default is an `-v` test rather than `:=`,
|
||||
which also fires on a set-but-empty value and would silently undo the rollback.
|
||||
|
||||
Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
|
||||
throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
|
||||
the `VkPhysicalDeviceMemoryProperties` heap sizes — no model, no load, no compute. The probe
|
||||
trap: `vulkaninfo --batch` is not a flag in Ubuntu 24.04's `vulkan-tools` — it prints the help
|
||||
page and exits 1, which reads like a driver failure. Every package involved has a candidate on
|
||||
the host's apt (`vulkan-tools` 1.4.341, `libvulkan-dev` 1.4.341, `glslang-dev`/`glslang-tools`
|
||||
16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older
|
||||
but present. Script: `/home/dank/spikes/npu-tts/vk_probe.sh` on Dankclaw.
|
||||
|
||||
Constraints that do not relax for a co-tenant: `/dev/dri` passthrough needs the ComfyUI
|
||||
pattern (`AddDevice` + `GroupAdd=keep-groups` + the world-writable udev rule) because
|
||||
systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at `/swap.img`
|
||||
exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents;
|
||||
`pp_dpm_fclk` reads `auto` (unheld), which only becomes a hazard if the NPU ever joins.
|
||||
|
||||
## 6. Re-visit condition
|
||||
|
||||
Retry the NPU when **a Linux loader claims the graph**: an `onnxruntime-genai-ryzenai`
|
||||
build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on `RyzenAI-npu5`, or that
|
||||
model appearing in a runtime's own supported catalog with a Linux runner. Then the
|
||||
prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices,
|
||||
the `ref_text` transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.
|
||||
Executable
+164
@@ -0,0 +1,164 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Destroy measurement ledgers in the surviving pages.
|
||||
|
||||
Point of use (AGENTS.md → Documentation Policy): a rate/throughput figure for
|
||||
anything not running and not being compared is destroyed, not filed. This pass
|
||||
deletes
|
||||
|
||||
1. any section whose heading is a bench/throughput/rate/ladder/verdict ledger,
|
||||
2. any table row or standalone bullet that REPORTS a rate (`123.4 t/s`,
|
||||
`tok/s`, `ms/token`, `TTFT`, acceptance %, `llama-bench`, a bench arm),
|
||||
|
||||
and it never touches a shape number (ctx, slots, MiB budgets, ports) or a bullet
|
||||
under a *Laws* / *Wire contract* / *Rollback* heading, where the sentence is the
|
||||
payload and the number is decoration. Dry run by default; `--apply` writes.
|
||||
|
||||
Usage: axe-measurements.py [--apply] [--verbose]
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
VAULT = pathlib.Path("serverAppConfigs-docs")
|
||||
RATE = re.compile(
|
||||
r"\d[\d,.]*\s*(t/s|tok/s|tokens?/s|tok s|tps|ms/token|µs/token|GB/s|GiB/s)\b|"
|
||||
r"\b(llama-bench|bench (arm|ladder|window)|decode ladder|prefill ladder)\b|"
|
||||
r"\bacceptance (rate|of)\b|\bdraft_n\b|\bttft\b|\bServer-Timing\b",
|
||||
re.I,
|
||||
)
|
||||
# A heading is a ledger only when the noun LEDS it and nothing instructional
|
||||
# follows: `## Throughput` yes, `## 6. The context ladder (find the max bootable
|
||||
# window by measurement)` no — that one is the method, which is point-of-use.
|
||||
LEDGER_HEAD = re.compile(
|
||||
r"^#{2,4}\s+(?:\d+[.) ]\s*)?(bench(es|mark|marks)?|throughput|results?|timings?|"
|
||||
r"measured (figures|numbers|results|rates|rates)|rates|verdict|acceptance|ladder)"
|
||||
r"\b[^|]{0,40}$",
|
||||
re.I,
|
||||
)
|
||||
INSTRUCTIONAL = re.compile(r"how|why|law|policy|method|procedure|find|by measurement|execution|flip|checklist", re.I)
|
||||
PROTECTED = re.compile(
|
||||
r"^#{2,4}\s+.*(law|wire|rollback|contract|shape|eras|files|health|see also|related|"
|
||||
r"boot|preflight|environment)",
|
||||
re.I,
|
||||
)
|
||||
def blank_rate_cells(text):
|
||||
"""Kill throughput COLUMNS wherever they appear: a cell that reports a rate
|
||||
becomes an em-dash, so the row keeps whatever else it records (the payload
|
||||
name, the reason, the rollback) and loses only the dead number."""
|
||||
out = []
|
||||
for line in text.split("\n"):
|
||||
st = line.strip()
|
||||
if st.startswith("|") and st.endswith("|") and RATE.search(line):
|
||||
cells = line.split("|")
|
||||
for i, c in enumerate(cells):
|
||||
if RATE.search(c):
|
||||
cells[i] = " — "
|
||||
line = "|".join(cells)
|
||||
out.append(line)
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def collapse_eras(text):
|
||||
"""One Eras row per era is fleet law; the NUMBERS in the old rows are not.
|
||||
Keep the current era verbatim, fold everything older into one row that says
|
||||
the numbers were destroyed and where the rollback actually lives (git)."""
|
||||
lines = text.split("\n")
|
||||
out = []
|
||||
in_eras = False
|
||||
ncol = 0
|
||||
folded = 0
|
||||
for line in lines:
|
||||
if line.startswith("#"):
|
||||
in_eras = bool(re.match(r"^#{2,4}\s+.*eras\b", line, re.I))
|
||||
if folded:
|
||||
out.append("| " + " | ".join(
|
||||
[f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured "
|
||||
"number destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
|
||||
folded = 0
|
||||
out.append(line)
|
||||
continue
|
||||
st = line.strip()
|
||||
if in_eras and st.startswith("|"):
|
||||
cells = [c.strip() for c in st.strip("|").split("|")]
|
||||
if set("".join(cells)) <= set(":- "):
|
||||
ncol = len(cells)
|
||||
out.append(line)
|
||||
continue
|
||||
if ncol == 0:
|
||||
ncol = len(cells)
|
||||
if re.search(r"2026-1[01]|current|active|running", st, re.I):
|
||||
out.append(line)
|
||||
else:
|
||||
folded += 1
|
||||
continue
|
||||
if folded and not st.startswith("|"):
|
||||
out.append("| " + " | ".join(
|
||||
[f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured number "
|
||||
"destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
|
||||
folded = 0
|
||||
ncol = 0
|
||||
out.append(line)
|
||||
if folded:
|
||||
out.append("| " + " | ".join(
|
||||
[f"≤ 2026-09 — {folded} earlier eras destroyed 2026-10-19"] + ["—"] * max(0, ncol - 1)))
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def main(argv):
|
||||
apply = "--apply" in argv
|
||||
verbose = "--verbose" in argv
|
||||
total_lines = 0
|
||||
pages = 0
|
||||
for page in sorted(VAULT.rglob("*.md")):
|
||||
s = str(page)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
text = page.read_text(errors="replace")
|
||||
before = len(text.split("\n"))
|
||||
if s.startswith("serverAppConfigs-docs/LLM/"):
|
||||
text = collapse_eras(blank_rate_cells(text))
|
||||
lines = text.split("\n")
|
||||
out = []
|
||||
skip = False
|
||||
under_protected = False
|
||||
cut = 0
|
||||
for line in lines:
|
||||
if line.startswith("#"):
|
||||
under_protected = bool(PROTECTED.search(line))
|
||||
skip = (bool(LEDGER_HEAD.match(line)) and not under_protected
|
||||
and not INSTRUCTIONAL.search(line))
|
||||
if skip:
|
||||
cut += 1
|
||||
continue
|
||||
if skip:
|
||||
cut += 1
|
||||
continue
|
||||
stripped = line.strip()
|
||||
out.append(line)
|
||||
if under_protected:
|
||||
# Eras rows are the rollback record, Law bullets are the payload:
|
||||
# the number is decoration inside them, and the row/bullet is not
|
||||
# a ledger entry. Leave the whole section alone.
|
||||
continue
|
||||
if stripped.startswith(("|", "-", "*")) and RATE.search(line):
|
||||
# Destroy only when the line EXISTS to report a number: a table
|
||||
# row, or a bullet whose verb is measurement.
|
||||
if stripped.startswith("|") or re.search(
|
||||
r"\b(measured|measures|bench|benched|ran at|runs at|clocked|hit|"
|
||||
r"landed at|came in at|peak|peaks at|sustained|wall.clock)\b", line, re.I):
|
||||
out.pop()
|
||||
cut += 1
|
||||
final = re.sub(r"\n{3,}", "\n\n", "\n".join(out))
|
||||
saved = before - len(final.split("\n"))
|
||||
if saved or cut:
|
||||
pages += 1
|
||||
total_lines += max(saved, cut)
|
||||
print(f" {s.replace('serverAppConfigs-docs/', '')}: -{max(saved, cut)}")
|
||||
if apply:
|
||||
page.write_text(final.rstrip("\n") + "\n")
|
||||
print(f"{'APPLIED' if apply else 'dry run'}: {pages} pages, {total_lines} lines of ledger")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Executable
+293
@@ -0,0 +1,293 @@
|
||||
#!/usr/bin/env python3
|
||||
"""OKF v0.1 conformance linter for a docs bundle (default serverAppConfigs-docs).
|
||||
|
||||
Checks the hard rules of the Open Knowledge Format spec plus the maintenance
|
||||
guidance, so "conforms to OKF" is a test and not a claim:
|
||||
|
||||
FRONTMATTER every non-reserved .md has parseable YAML frontmatter with a
|
||||
non-empty `type` from the spec's value set
|
||||
TYPE the value is one of reference / howto / tutorial / explanation
|
||||
DESC `description` is present, non-blank, not the title, <= 120 chars,
|
||||
and does not restate a fleet quantity
|
||||
TIMESTAMP `timestamp` present, ISO 8601, not in the future
|
||||
LENGTH lines between 1 and 500 inclusive
|
||||
INDEX-FM no index.md may carry frontmatter (index.md is reserved)
|
||||
LOG-DATE log.md date headings are `## YYYY-MM-DD` or `## YYYY-MM-DD — Title`,
|
||||
one entry per date, newest first
|
||||
INDEX-COVER every index.md links each of its sibling pages; the root index
|
||||
links every page in the bundle
|
||||
INDEX-NUM an index table row restates a fleet quantity — a digit run of
|
||||
three or more, a digits-plus-unit figure, a port, or a
|
||||
multiplier. Names (Qwen3.6, RDNA4, RTX 3090, PICO-8, P4) are
|
||||
not quantities and are allowed
|
||||
CITATIONS informational: pages with factual claims and no Citations
|
||||
section are counted, never failed
|
||||
|
||||
LENGTH lines between 1 and 500 inclusive. A page may opt out of the
|
||||
ceiling with `okf-exempt: length` in its frontmatter: AGENTS.md
|
||||
makes one page per endpoint permanent, and an endpoint page with
|
||||
a full Eras table cannot split without creating a second owner
|
||||
of the same shape (tests/docs_completeness.bats enforces that)
|
||||
|
||||
Exit 0 when there are no violations.
|
||||
"""
|
||||
import datetime
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
TYPES = {"reference", "howto", "tutorial", "explanation"}
|
||||
RESERVED = {"index.md", "log.md", "AGENTS.md"}
|
||||
|
||||
# A quantity, as opposed to a name: a comma-grouped figure, a run of 3+ digits,
|
||||
# a unit-bearing figure, a port (":7770"), or a multiplier ("2x", "3 *").
|
||||
QUANTITY = re.compile(
|
||||
r"\d{1,3}(?:,\d{3})+" # 262,144
|
||||
r"|\d{3,}" # 200000, 7770, 3090
|
||||
r"|\d+(?:\.\d+)?\s*(?:GiB|MiB|GB|MB|TB|tok/s|t/s|x\b|%)"
|
||||
r"|[:×]\s*\d"
|
||||
r"|\d+\s*[x×]\s*\d"
|
||||
)
|
||||
# Product / model / standard names that legitimately contain digits.
|
||||
NAMES = re.compile(
|
||||
r"Qwen[\d.]*[-\w]*|Gemma\s?\d[\w.]*|MiMo[-\w.\d.]*|Ornith[-\w.\d]*"
|
||||
r"|Bonsai[-\w.\d]*|Ling[-\w.\d.]*|DeepSeek[-\w.\d]*|GLM[-\w.\d]*"
|
||||
r"|PICO-\d+|RDNA\s?\d+|R\d{4}|RTX\s?\d+|XDNA\d|gfx\d+|CUDA\s?[\d.]+"
|
||||
r"|\d+\s?GbE|Wi-?Fi\s?\d|v?\d+\.\d+(?:\.\d+)?|b\d+|P\d\b|A\d+\b"
|
||||
r"|frank-\w*2026-\d\d-\d\d|2026-\d\d-\d\d",
|
||||
re.I,
|
||||
)
|
||||
DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}")
|
||||
FM_KEY = re.compile(r"^([A-Za-z][\w-]*):\s*(.*)$")
|
||||
|
||||
|
||||
def split_frontmatter(text):
|
||||
if not text.startswith("---"):
|
||||
return None, text
|
||||
end = text.find("\n---", 3)
|
||||
if end < 0:
|
||||
return None, text
|
||||
body_start = text.find("\n", end + 1)
|
||||
return text[3:end], text[body_start + 1 :] if body_start >= 0 else ""
|
||||
|
||||
|
||||
def fields(raw):
|
||||
out = {}
|
||||
if not raw:
|
||||
return out
|
||||
key = None
|
||||
for line in raw.splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
m = FM_KEY.match(line)
|
||||
if m:
|
||||
key, val = m.group(1), m.group(2).strip().strip("\"'")
|
||||
out[key] = val
|
||||
elif line.startswith(" ") and key:
|
||||
out[key] = (out[key] + " " + line.strip()).strip()
|
||||
return out
|
||||
|
||||
|
||||
def strip_names(value):
|
||||
return NAMES.sub(" ", value)
|
||||
|
||||
|
||||
def check_file(path: Path, root: Path, length_exempt=frozenset()):
|
||||
v = []
|
||||
rel = str(path.relative_to(root))
|
||||
text = path.read_text(errors="replace")
|
||||
lines = text.splitlines()
|
||||
name = path.name
|
||||
raw, body = split_frontmatter(text)
|
||||
fm = fields(raw)
|
||||
|
||||
if name not in RESERVED:
|
||||
if raw is None:
|
||||
v.append(("FRONTMATTER", rel, "no parseable frontmatter"))
|
||||
else:
|
||||
t = fm.get("type", "")
|
||||
if not t:
|
||||
v.append(("TYPE", rel, "missing type"))
|
||||
elif t not in TYPES:
|
||||
v.append(("TYPE", rel, f"type '{t}' not in {sorted(TYPES)}"))
|
||||
d = fm.get("description", "")
|
||||
if not d:
|
||||
v.append(("DESC", rel, "missing description"))
|
||||
else:
|
||||
if len(d) > 120:
|
||||
v.append(("DESC", rel, f"description is {len(d)} chars (max 120)"))
|
||||
if d.strip("# ").lower() == fm.get("title", "\0").strip("# ").lower():
|
||||
v.append(("DESC", rel, "description equals title"))
|
||||
if QUANTITY.search(strip_names(d)):
|
||||
v.append(("DESC", rel, "description restates a quantity"))
|
||||
ts = fm.get("timestamp", "")
|
||||
if not ts:
|
||||
v.append(("TIMESTAMP", rel, "missing timestamp"))
|
||||
else:
|
||||
try:
|
||||
stamp = datetime.date.fromisoformat(ts[:10])
|
||||
if stamp > datetime.date.today():
|
||||
v.append(("TIMESTAMP", rel, f"timestamp {ts} is in the future"))
|
||||
except ValueError:
|
||||
v.append(("TIMESTAMP", rel, f"unparseable timestamp {ts!r}"))
|
||||
|
||||
if name == "index.md" and raw is not None:
|
||||
v.append(("INDEX-FM", rel, "index.md must not carry frontmatter"))
|
||||
|
||||
if name == "log.md":
|
||||
dates = []
|
||||
for line in lines:
|
||||
m = re.match(r"^##\s+(\S.*?)\s*$", line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1)
|
||||
if not DATE_RE.match(head):
|
||||
continue
|
||||
dates.append(head[:10])
|
||||
# OKF requires the heading to USE the date format; a title after an
|
||||
# em-dash is allowed and costs zero lines, so `## 2026-08-17 — Why the
|
||||
# pair was rebuilt` is conformant and `## 2026-08-17Rebuilt` is not.
|
||||
if head != head[:10] and not re.match(r"^\d{4}-\d{2}-\d{2}\s+—\s+\S", head):
|
||||
v.append(("LOG-DATE", rel,
|
||||
f"date heading must be the date alone or `date — Title`: '## {head}'"))
|
||||
if len(dates) != len(set(dates)):
|
||||
v.append(("LOG-DATE", rel, "duplicate date heading"))
|
||||
if dates != sorted(dates, reverse=True):
|
||||
v.append(("LOG-DATE", rel, "date headings not newest first"))
|
||||
|
||||
if raw is not None and name not in RESERVED:
|
||||
n = len(lines)
|
||||
body_lines = len([ln for ln in body.splitlines() if ln.strip()])
|
||||
opted_out = "length" in fm.get("okf-exempt", "").lower()
|
||||
if n > 500:
|
||||
if rel in length_exempt:
|
||||
v.append(("LENGTH-OK", rel,
|
||||
f"{n} lines — over the hint, pinned by the "
|
||||
"one-page-per-endpoint law"))
|
||||
elif opted_out:
|
||||
v.append(("LENGTH-OK", rel,
|
||||
f"{n} lines — over the hint, opted out by the page"))
|
||||
else:
|
||||
v.append(("LENGTH", rel, f"{n} lines (max 500)"))
|
||||
elif body_lines < 15:
|
||||
v.append(("LENGTH", rel, "stub: fewer than 15 body lines"))
|
||||
|
||||
# A second `type:` key inside the body means the page carries two copies of
|
||||
# itself with a frontmatter block in the middle — the shape a naive
|
||||
# frontmatter rewrite produces. It doubles the length, splits the wikilinks
|
||||
# and makes the page's facts ambiguous, so it is a hard failure.
|
||||
if raw is not None and name not in RESERVED:
|
||||
dupes = [i + len((raw or "").splitlines()) + 2
|
||||
for i, ln in enumerate(body.splitlines())
|
||||
if re.match(r"^(type|title|description):", ln)]
|
||||
if dupes:
|
||||
v.append(("DOUBLED", rel,
|
||||
f"frontmatter keys inside the body at line(s) "
|
||||
f"{', '.join(str(d) for d in dupes[:5])} — the page "
|
||||
"contains a duplicated copy of itself"))
|
||||
|
||||
return v
|
||||
|
||||
|
||||
def index_rows(text):
|
||||
rows = []
|
||||
for line in text.splitlines():
|
||||
if line.startswith("|") and "[[" in line:
|
||||
rows.append(line)
|
||||
return rows
|
||||
|
||||
|
||||
def check_links(root: Path):
|
||||
v = []
|
||||
pages = [
|
||||
p for p in sorted(root.rglob("*.md"))
|
||||
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
|
||||
]
|
||||
for index in [p for p in pages if p.name == "index.md"]:
|
||||
text = index.read_text(errors="replace")
|
||||
links = set(re.findall(r"\[\[([^\]|#]+)", text))
|
||||
if index.parent == root:
|
||||
targets = [p for p in pages if p.name not in RESERVED]
|
||||
else:
|
||||
targets = [p for p in index.parent.glob("*.md") if p.name != "index.md"]
|
||||
for target in targets:
|
||||
if target.stem not in links and target.name not in links:
|
||||
v.append(("INDEX-COVER", str(index.relative_to(root)),
|
||||
f"does not link sibling {target.name}"))
|
||||
for index in [p for p in pages if p.name == "index.md"]:
|
||||
for row in index_rows(index.read_text(errors="replace")):
|
||||
if QUANTITY.search(strip_names(row)):
|
||||
v.append(("INDEX-NUM", str(index.relative_to(root)),
|
||||
"row restates a quantity: " + row[:80]))
|
||||
return v
|
||||
|
||||
|
||||
def live_endpoints(root: Path):
|
||||
"""Pages pinned by AGENTS.md's one-page-per-endpoint law.
|
||||
|
||||
Derived, never hardcoded — the same derivation as
|
||||
tests/docs_completeness.bats: an endpoint is live when scripts/ holds both
|
||||
its env and its unit. Those pages carry the Eras table for their whole
|
||||
history, so the OKF 500-line split hint cannot apply to them without
|
||||
creating a second owner of the same shape.
|
||||
"""
|
||||
scripts = root.parent / "scripts"
|
||||
out = set()
|
||||
if scripts.is_dir():
|
||||
for envf in scripts.glob("frank-*.env"):
|
||||
if (scripts / f"{envf.stem}.service").exists():
|
||||
out.add(f"LLM/{envf.stem}.md")
|
||||
return out
|
||||
|
||||
|
||||
def retired_era_pages(root: Path):
|
||||
"""Era pages of a retired endpoint: campaign ledgers, one owner each.
|
||||
|
||||
AGENTS.md lets a campaign ledger that exists nowhere else live on the page
|
||||
that measured it, and a retired endpoint keeps exactly one page per era.
|
||||
Splitting such a ledger to satisfy the 500-line hint would fragment one
|
||||
campaign across two owners, so the tag opts the page out.
|
||||
"""
|
||||
out = set()
|
||||
for path in root.rglob("*.md"):
|
||||
s = str(path)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
raw, _ = split_frontmatter(path.read_text(errors="replace"))
|
||||
if raw is not None and "retired-endpoint" in fields(raw).get("tags", ""):
|
||||
out.add(str(path.relative_to(root)))
|
||||
return out
|
||||
|
||||
|
||||
def main(argv):
|
||||
root = Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
|
||||
exempt = frozenset(live_endpoints(root)) | retired_era_pages(root)
|
||||
violations = []
|
||||
notes = []
|
||||
citation_gaps = 0
|
||||
pages = 0
|
||||
for path in sorted(root.rglob("*.md")):
|
||||
s = str(path)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
pages += 1
|
||||
for code, rel, detail in check_file(path, root, exempt):
|
||||
(notes if code == "LENGTH-OK" else violations).append((code, rel, detail))
|
||||
text = path.read_text(errors="replace")
|
||||
if len(text.splitlines()) > 40 and not re.search(
|
||||
r"^##\s+(Citations|Sources)\b", text, re.M):
|
||||
citation_gaps += 1
|
||||
violations += check_links(root)
|
||||
|
||||
for code, rel, detail in violations:
|
||||
print(f" {code:12s} {rel}: {detail}")
|
||||
for code, rel, detail in notes:
|
||||
print(f" {code:12s} {rel}: {detail}")
|
||||
print(f"{pages} concept pages, {len(violations)} OKF violations"
|
||||
f" ({citation_gaps} pages without a Citations section — soft;"
|
||||
f" {len(notes)} length exemptions honoured)")
|
||||
return 1 if violations else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -31,7 +31,12 @@ def strip_md(path: str) -> str:
|
||||
|
||||
|
||||
def build_index(vault: pathlib.Path):
|
||||
pages = sorted(vault.rglob("*.md"))
|
||||
# `.AGENTTRASH` is deleted-but-held state, not vault content: it must not be
|
||||
# scanned (its relative links resolve against the trash dir, not the vault)
|
||||
# and must not answer as a link target — a live page pointing into the trash
|
||||
# is an unresolved link, not a resolved one.
|
||||
pages = sorted(p for p in vault.rglob("*.md")
|
||||
if ".AGENTTRASH" not in str(p))
|
||||
stems: dict[str, list[pathlib.Path]] = {}
|
||||
for page in pages:
|
||||
stems.setdefault(page.stem.casefold(), []).append(page)
|
||||
|
||||
@@ -293,6 +293,17 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
if len(parts) >= 2 and parts[1] in {"caddy", "comfyui", "loki", "alloy", "hermes", "anubis", "seerr"}:
|
||||
_add(services, parts[1])
|
||||
return services
|
||||
if len(parts) >= 3 and parts[1] == "host":
|
||||
# Host state a service needs at boot lives under dankclaw/host/.
|
||||
# The fabric-clock hold is frank-dragoon's NPU precondition (the
|
||||
# engine refuses to start with HALOGEN_NPU_MODELS set while the
|
||||
# clock moves) and the deployer installs and gates it, so changing
|
||||
# it re-dispatches that endpoint. `grub` is the tracked boot line,
|
||||
# which moves only through scripts/set-dankclaw-bootline.sh and is
|
||||
# a reboot decision — it plans no service.
|
||||
if parts[2].startswith("halogen-fabric-clock"):
|
||||
_add(services, "frank-dragoon")
|
||||
return services
|
||||
if len(parts) >= 4 and parts[1] == "authentik" and parts[2] == "quadlet":
|
||||
name = parts[3].removesuffix(".container")
|
||||
if name in {"authentik-postgres", "authentik-server", "authentik-worker", "authentik-outpost", "authentik-outpost-ldap"}:
|
||||
|
||||
@@ -1329,6 +1329,9 @@ deploy_anubis() {
|
||||
# port gate is the same as every cutover since 2026-09-16: nothing
|
||||
# starts until 7770 is released.
|
||||
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
||||
# The payload dir is its own twin because the NPU lane's containment gate below
|
||||
# reads it; every artifact path under it stays a full literal as usual.
|
||||
LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}"
|
||||
LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-v2-abliterated.hgn}"
|
||||
LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678432}"
|
||||
LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-ngram.hgn}"
|
||||
@@ -1337,6 +1340,18 @@ LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-v2-abliterat
|
||||
LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
|
||||
LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-v2-abliterated/tokenizer}"
|
||||
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
||||
# NPU lane (2026-10-19). HALOGEN_NPU_MODELS is the whole switch and the parity
|
||||
# twin of the env file's line: empty here AND in the env = the pre-lane
|
||||
# endpoint. The gates below mirror serve-frank-dragoon.sh, and the vendor's
|
||||
# fabric-clock host unit (dankclaw/host/, verbatim at tag v0.16.2) is installed
|
||||
# ONLY while the lane is named — on Strix Halo the iGPU and the NPU co-running
|
||||
# hang the machine while the fabric clock moves, so halogen refuses to boot with
|
||||
# the lane armed unless that clock is held.
|
||||
LLM_DRAGOON_NPU_MODELS="${LLM_DRAGOON_NPU_MODELS:-qwen3-embedding-0.6b}"
|
||||
LLM_DRAGOON_NPU_DIR="${LLM_DRAGOON_NPU_DIR:-/home/dank/models/halogen-v2-abliterated/npu}"
|
||||
LLM_DRAGOON_DEV_ACCEL="${LLM_DRAGOON_DEV_ACCEL:-/dev/accel/accel0}"
|
||||
LLM_DRAGOON_XRT_DIR="${LLM_DRAGOON_XRT_DIR:-/usr/lib/x86_64-linux-gnu}"
|
||||
LLM_DRAGOON_NETWORK="${LLM_DRAGOON_NETWORK:-dankclaw_default}"
|
||||
|
||||
retire_halogen_dragoon_quadlet() {
|
||||
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
|
||||
@@ -1415,6 +1430,63 @@ deploy_llm_dragoon() {
|
||||
fi
|
||||
done
|
||||
|
||||
# The container network has to exist before the run line names it (a typo
|
||||
# there is a restart-looping unit, not a config error anyone reads).
|
||||
if [[ -n "$LLM_DRAGOON_NETWORK" ]] && ! podman network exists "$LLM_DRAGOON_NETWORK"; then
|
||||
echo "ERROR: podman network $LLM_DRAGOON_NETWORK does not exist" >&2
|
||||
echo "frank-dragoon: create it or blank FRANK_DRAGOON_NETWORK (pasta default)" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# --- NPU lane: the same predicates the serve script checks --------------
|
||||
if [[ -n "$LLM_DRAGOON_NPU_MODELS" ]]; then
|
||||
# One door into the container: the read-only /models mount of the
|
||||
# payload dir. A staging dir outside it needs a nested bind whose
|
||||
# mountpoint runc cannot create through a read-only parent, so the
|
||||
# parity twin of the serve gate refuses the shape before the restart.
|
||||
if [[ "$LLM_DRAGOON_NPU_DIR/" != "$LLM_DRAGOON_MODEL_DIR"/* ]]; then
|
||||
echo "ERROR: NPU staging dir $LLM_DRAGOON_NPU_DIR is outside the payload dir $LLM_DRAGOON_MODEL_DIR — the lane's files reach the container only through the read-only /models mount" >&2
|
||||
return 1
|
||||
fi
|
||||
if [[ ! -e "$LLM_DRAGOON_DEV_ACCEL" ]]; then
|
||||
echo "ERROR: NPU node $LLM_DRAGOON_DEV_ACCEL absent — amdxdna is not bound (is iommu=pt still on the boot line?)" >&2
|
||||
return 1
|
||||
fi
|
||||
for xl in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
|
||||
if [[ ! -e "$LLM_DRAGOON_XRT_DIR/$xl" ]]; then
|
||||
echo "ERROR: XRT NPU library $LLM_DRAGOON_XRT_DIR/$xl absent (Ubuntu libxrt2 + libxrt-npu2)" >&2
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
for nm in ${LLM_DRAGOON_NPU_MODELS//,/ }; do
|
||||
if [[ ! -d "$LLM_DRAGOON_NPU_DIR/$nm" ]]; then
|
||||
echo "ERROR: NPU model $nm not staged in $LLM_DRAGOON_NPU_DIR" >&2
|
||||
echo "frank-dragoon: stage it: bash scripts/download-frank-dragoon-npu.sh" >&2
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
# The hold must exist BEFORE the engine starts and survive every boot
|
||||
# (a resume hands the clock back too — the unit's WantedBy covers it).
|
||||
run_privileged install -m 0755 dankclaw/host/halogen-fabric-clock \
|
||||
/usr/local/sbin/halogen-fabric-clock
|
||||
run_privileged install -m 0644 dankclaw/host/halogen-fabric-clock.service \
|
||||
/etc/systemd/system/halogen-fabric-clock.service
|
||||
run_privileged systemctl daemon-reload
|
||||
run_privileged systemctl enable --now halogen-fabric-clock.service
|
||||
if ! run_privileged /usr/local/sbin/halogen-fabric-clock status; then
|
||||
echo "ERROR: fabric clock is not held — halogen refuses to start with HALOGEN_NPU_MODELS set" >&2
|
||||
echo "frank-dragoon: journalctl -u halogen-fabric-clock.service -n 40" >&2
|
||||
return 1
|
||||
fi
|
||||
elif [[ -x /usr/local/sbin/halogen-fabric-clock ]]; then
|
||||
# Lane rolled back: hand the clock back to the driver and drop the boot
|
||||
# hook. `disable` deletes the INSTALLED copy of this unit — unlike the
|
||||
# tracked LLM unit symlinks the fleet's disable law protects, this file
|
||||
# is repo-owned and the next lane-on deploy reinstalls it.
|
||||
run_privileged systemctl disable halogen-fabric-clock.service 2>/dev/null || true
|
||||
run_privileged /usr/local/sbin/halogen-fabric-clock release || true
|
||||
fi
|
||||
|
||||
retire_halogen_dragoon_quadlet
|
||||
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
||||
# the container) before waiting on the port. The stray sweep is keyed
|
||||
@@ -1434,6 +1506,33 @@ deploy_llm_dragoon() {
|
||||
run_privileged systemctl enable frank-dragoon.service
|
||||
run_privileged systemctl restart frank-dragoon.service
|
||||
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
|
||||
# The lane's own gate. /health proves the engine is up, nothing about the
|
||||
# NPU programs; one real embedding is the observable output. The first
|
||||
# request after a boot is a warmup artifact (fleet law), so retry before
|
||||
# calling a booted lane dead.
|
||||
if [[ -n "$LLM_DRAGOON_NPU_MODELS" ]]; then
|
||||
for nm in ${LLM_DRAGOON_NPU_MODELS//,/ }; do
|
||||
lane_ok=0
|
||||
for _try in 1 2 3 4 5; do
|
||||
if curl -fsS -m 60 -H 'content-type: application/json' \
|
||||
-d "{\"model\":\"$nm\",\"input\":[\"frank-dragoon npu lane gate\"]}" \
|
||||
http://127.0.0.1:7770/v1/embeddings 2>/dev/null \
|
||||
| python3 -c 'import json,sys
|
||||
v = json.load(sys.stdin)["data"][0]["embedding"]
|
||||
raise SystemExit(0 if isinstance(v, list) and len(v) and all(isinstance(x, (int, float)) for x in v) else 1)'; then
|
||||
lane_ok=1
|
||||
break
|
||||
fi
|
||||
sleep 10
|
||||
done
|
||||
if [[ "$lane_ok" != 1 ]]; then
|
||||
echo "ERROR: NPU model $nm answered no vector on /v1/embeddings" >&2
|
||||
echo "frank-dragoon: read the boot ledger: tail -100 /home/dank/logs/frank_dragoon.log" >&2
|
||||
return 1
|
||||
fi
|
||||
echo "frank-dragoon: NPU lane verified — $nm embedded"
|
||||
done
|
||||
fi
|
||||
}
|
||||
|
||||
# deploy_llm_zealot — frank-zealot is llama.cpp (PrismML fork build) on the
|
||||
@@ -1463,6 +1562,12 @@ LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-prism-b10
|
||||
# 868.32 MiB worst-case VRAM (595.29 model + 248.10 compute in the boot ledger)
|
||||
# lands AFTER the engine's fit check, so it is not inside the q8_0 budget.
|
||||
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf}"
|
||||
# The refusal adapter is part of the payload identity now, so the deployer
|
||||
# checks it beside the other two files — but it has its OWN fetcher, so a
|
||||
# missing adapter must not send the operator to the bonsai9 script. Integrity
|
||||
# (bytes + sha256) is the fetcher's job at download time; this gate only asks
|
||||
# whether the file the env names exists, exactly like the two above.
|
||||
LLM_ZEALOT_LORA="${LLM_ZEALOT_LORA:-/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf}"
|
||||
|
||||
deploy_llm_zealot() {
|
||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||
@@ -1484,6 +1589,13 @@ deploy_llm_zealot() {
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
if [[ ! -f "$LLM_ZEALOT_LORA" ]]; then
|
||||
echo "ERROR: frank-zealot refusal adapter absent at $LLM_ZEALOT_LORA" >&2
|
||||
echo "frank-zealot: run scripts/download-frank-zealot-uncensored-lora.sh first;" >&2
|
||||
echo "frank-zealot: the env names the adapter, so the unit would refuse to boot" >&2
|
||||
echo "frank-zealot: without it — starting the unit anyway is a crash loop." >&2
|
||||
return 1
|
||||
fi
|
||||
relink_unit frank-zealot
|
||||
run_privileged systemctl daemon-reload
|
||||
run_privileged systemctl enable frank-zealot.service
|
||||
|
||||
Executable
+109
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Destroy the vault's dead weight: era ledgers, retired runtimes, dead experiments.
|
||||
|
||||
`AGENTS.md` was treating a measurement as a record worth keeping forever. It is not: a
|
||||
throughput figure for a payload that no longer runs, on a runtime that no longer exists,
|
||||
is a liability — the next agent spends context on a number that AI progress has already
|
||||
obsolete. This pass moves those pages to `.AGENTTRASH/` (repo law: never `rm`) and then
|
||||
deletes every reference to them from the surviving pages, including the sentence that
|
||||
carried the link when the link was the sentence's only content.
|
||||
|
||||
Usage: destroy-vault.py <list-file>
|
||||
List file: one vault-relative path per line; a directory path destroys the whole tree.
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
VAULT = pathlib.Path("serverAppConfigs-docs")
|
||||
TRASH = VAULT / ".AGENTTRASH" / "20261019-destruction"
|
||||
|
||||
|
||||
def targets(list_file):
|
||||
out = []
|
||||
for line in pathlib.Path(list_file).read_text().splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
p = VAULT / line
|
||||
if p.is_dir():
|
||||
out += [f for f in sorted(p.rglob("*.md"))]
|
||||
elif p.is_file():
|
||||
out.append(p)
|
||||
else:
|
||||
print(f" SKIP (missing): {line}")
|
||||
return out
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 2:
|
||||
print(__doc__)
|
||||
return 2
|
||||
doomed = targets(argv[1])
|
||||
# Stems come from the trash too, so re-running the pass (or running it after a
|
||||
# partial destruction) still knows which names are dead and keeps scrubbing.
|
||||
stems = {p.stem for p in doomed}
|
||||
if TRASH.is_dir():
|
||||
stems |= {p.stem for p in TRASH.rglob("*.md")}
|
||||
moved = 0
|
||||
for p in doomed:
|
||||
dest = TRASH / p.relative_to(VAULT)
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.move(str(p), str(dest))
|
||||
moved += 1
|
||||
print(f"destroyed {moved} pages ({sum(len(p.read_text(errors='replace').splitlines()) for p in TRASH.rglob('*.md'))} lines staged in trash)")
|
||||
|
||||
# Repoint-or-delete: a link to a destroyed page is itself dead information.
|
||||
link_wl = re.compile(r"\[\[([^\]|#]+)(?:#[^\]|]*)?(?:\|([^\]]*))?\]\]")
|
||||
link_md = re.compile(r"\[[^\]]*\]\(([^)#]*\.md)(?:#[^)]*)?\)")
|
||||
pages_cleaned = links_dropped = lines_dropped = 0
|
||||
for page in sorted(VAULT.rglob("*.md")):
|
||||
s = str(page)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
text = page.read_text(errors="replace")
|
||||
lines = text.split("\n")
|
||||
out = []
|
||||
hits = 0
|
||||
|
||||
def dead(target):
|
||||
# NOT pathlib.Path(...).stem: that strips a real suffix off names like
|
||||
# `Lemonade and sd.cpp`, so a link to a destroyed page survives and the
|
||||
# vault then fails its own link check.
|
||||
t = target.strip()
|
||||
if t.endswith(".md"):
|
||||
t = t[:-3]
|
||||
return t in stems or pathlib.PurePosixPath(t).name in stems
|
||||
|
||||
for line in lines:
|
||||
found = [t for t in link_wl.findall(line) if dead(t[0])]
|
||||
for m in link_md.finditer(line):
|
||||
if dead(m.group(1)):
|
||||
found.append(m.group(1))
|
||||
if not found:
|
||||
out.append(line)
|
||||
continue
|
||||
hits += len(found)
|
||||
stripped = link_wl.sub(
|
||||
lambda m: m.group(2) if (m.group(2) and not dead(m.group(1))) else "",
|
||||
line)
|
||||
stripped = link_md.sub(
|
||||
lambda m: "" if dead(m.group(1)) else m.group(0), stripped)
|
||||
# What is left of the line after the dead links are gone?
|
||||
residue = re.sub(r"[\s\-*•·|:;,.\u2014\u2013]+", "", stripped)
|
||||
if residue:
|
||||
out.append(re.sub(r"\s+$", "", re.sub(r"^(\s*[-*]\s*)[\s:;,.\u2014-]+", r"\1", stripped)))
|
||||
else:
|
||||
lines_dropped += 1
|
||||
if hits:
|
||||
page.write_text("\n".join(out))
|
||||
pages_cleaned += 1
|
||||
links_dropped += hits
|
||||
print(f"cleaned {pages_cleaned} surviving pages: {links_dropped} dead links dropped, "
|
||||
f"{lines_dropped} lines left empty and deleted")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Executable
+117
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env bash
|
||||
# download-frank-dragoon-npu.sh — stage the NPU half of frank-dragoon: the
|
||||
# vendor-precompiled AIE programs halogen-flash-server runs on the XDNA2 NPU
|
||||
# when HALOGEN_NPU_MODELS names them. These are NOT model weights you can
|
||||
# quantize yourself: XDNA2 executes only programs compiled by AMD's AIE
|
||||
# compiler, so the only source is peonist-ai's per-model NPU repo, and the
|
||||
# engine checks every file's size and sha256 against its own catalog at boot
|
||||
# (/opt/halogen/npu/models.txt inside the image) and refuses to load a model
|
||||
# whose files do not match.
|
||||
#
|
||||
# The manifest below IS that catalog block (dumped from the pinned 0.16.2
|
||||
# image on 2026-10-19), so the files land in the layout the entrypoint
|
||||
# expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and
|
||||
# <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep
|
||||
# /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD
|
||||
# stays unset; the vendor's own README: with it unset the container makes no
|
||||
# outbound connections at all).
|
||||
#
|
||||
# The fetch law, same as every sibling script: curl -f (an access error must
|
||||
# not become a "successful" 153-byte artifact), the exact byte count and the
|
||||
# sha256 from the manifest, --verify-only to re-check a staged set.
|
||||
#
|
||||
# Adding a second NPU model: read its block out of the image's catalog and add
|
||||
# a case below —
|
||||
# podman exec frank-dragoon-halogen grep "^file <model-id> " /opt/halogen/npu/models.txt
|
||||
# podman exec frank-dragoon-halogen grep "^model <model-id> " /opt/halogen/npu/models.txt
|
||||
# and add the model id to HALOGEN_NPU_MODELS in frank-dragoon.env.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||
# The env file owns the staging path (it must sit inside the payload dir, which
|
||||
# is the container's only /models mount); an exported FRANK_DRAGOON_NPU_DIR wins
|
||||
# so the bats seam can stage somewhere disposable.
|
||||
DEST="${FRANK_DRAGOON_NPU_DIR:-}"
|
||||
if [ -z "$DEST" ] && [ -f "$ENV_FILE" ]; then
|
||||
DEST="$(sed -nE 's/^FRANK_DRAGOON_NPU_DIR=(.*)$/\1/p' "$ENV_FILE" | tail -1)"
|
||||
fi
|
||||
: "${DEST:?FRANK_DRAGOON_NPU_DIR is not set and $ENV_FILE names no FRANK_DRAGOON_NPU_DIR}"
|
||||
HF="https://huggingface.co"
|
||||
|
||||
MODELS="${HALOGEN_NPU_MODELS:-}"
|
||||
if [ -z "$MODELS" ] && [ -f "$ENV_FILE" ]; then
|
||||
MODELS="$(sed -nE 's/^HALOGEN_NPU_MODELS=(.*)$/\1/p' "$ENV_FILE" | tail -1)"
|
||||
fi
|
||||
|
||||
VERIFY_ONLY=0
|
||||
case "${1:-}" in
|
||||
--verify-only) VERIFY_ONLY=1 ;;
|
||||
"") ;;
|
||||
*) echo "usage: ${0##*/} [--verify-only] (models come from HALOGEN_NPU_MODELS)" >&2; exit 2 ;;
|
||||
esac
|
||||
[ -n "$MODELS" ] || { echo "no NPU model named — set HALOGEN_NPU_MODELS or run with it in the environment" >&2; exit 1; }
|
||||
|
||||
# manifest <model-id> -> repo revision + one "path bytes sha256" per file.
|
||||
manifest() {
|
||||
case "$1" in
|
||||
qwen3-embedding-0.6b)
|
||||
NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b"
|
||||
NPU_REV="4960fd9d4423dd15373ad21c83db48dbdb3b2798"
|
||||
FILES=(
|
||||
"devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91"
|
||||
"devices/consts.hnpb 5086496 ae430e41545a0b1012938cda373b0c450364d3be631111cca3a8c43679f93336"
|
||||
"devices/devices.hnpm 24047 4e9bca9d95d084fb89ce5b99f579a00666c8ca1efae2613c53e3a4878cc9f174"
|
||||
"devices/u0.elf 4931968 b38f165886cb037b7d3951d9cb878473eaba58a432acd45416ff85fd88aa4e46"
|
||||
"devices/u1.elf 5456000 b1811feee851b8af833b37721da92099d89772110815bbbe2c3fa1c276f0ff42"
|
||||
"devices/u2.elf 6505552 406ea9a95b470628a7382e376f4a72cdf4d7aa4c59b4ddf0a4bdd139d63d6ff2"
|
||||
"devices/u3.elf 6817216 bc205d51b523d770151a0c61968f7c70bf624412827580bb6fe0aaf4650dc2ee"
|
||||
"devices/u4.elf 7444032 a5a91338a2603acdcd5c11df532033fda2d3f5479ea44ae87f98fafbdff9a62d"
|
||||
"devices/u5.elf 7432416 6c2bf41835569e0eba25ded7c05c7264ae69c6700bdeb4791057d884f2a4c126"
|
||||
"devices/u6.elf 4941232 919d0038ca088eafddf195f3466ff38db27d3635be1cffa5d4d4b5140902a68e"
|
||||
"devices/u7.elf 5488304 d904cf8afae9428dd167960549e18cce5441745f18c62f8c50975dffc077d3c2"
|
||||
"devices/u8.elf 4941232 db3aef69b17825854ee0dd2df475125a3f5c872f95c686793d6af1ee83596fa7"
|
||||
"devices/u9.elf 5488304 d182727e551084db8fb9600441d8aa373c7f09a3d267ae0a5e921ef92326db1c"
|
||||
"qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1"
|
||||
"tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a"
|
||||
) ;;
|
||||
*)
|
||||
echo "unknown NPU model '$1' — add its catalog block to manifest() in this script" >&2
|
||||
return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
for MODEL in ${MODELS//,/ }; do
|
||||
manifest "$MODEL" || exit 1
|
||||
echo "== $MODEL ($NPU_REPO@$NPU_REV) =="
|
||||
for entry in "${FILES[@]}"; do
|
||||
read -r f want sha <<<"$entry"
|
||||
path="$DEST/$MODEL/$f"
|
||||
mkdir -p "$(dirname "$path")"
|
||||
if [ "$VERIFY_ONLY" -eq 0 ]; then
|
||||
if [ -f "$path" ] && [ "$(stat -c %s "$path")" = "$want" ]; then
|
||||
echo "HAVE $f ($want bytes) — skipping fetch"
|
||||
else
|
||||
echo "FETCH $f -> $path"
|
||||
# HF answers /resolve/ with a 302 to the Xet CDN and curl
|
||||
# DROPS the Range header across redirects, so resolve the
|
||||
# effective URL first and resume against the CDN (the sibling
|
||||
# 62 GiB fetch hit this on 2026-10-18).
|
||||
url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \
|
||||
|| { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; }
|
||||
curl -fL --retry 5 -C - -o "$path" "$url"
|
||||
fi
|
||||
fi
|
||||
[ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
|
||||
if [ -L "$path" ]; then
|
||||
echo "SYMLINK $path — the serve gate refuses symlinks; replace with a real file" >&2
|
||||
exit 1
|
||||
fi
|
||||
got="$(stat -c %s "$path")"
|
||||
[ "$got" = "$want" ] || { echo "SIZE $path: $got != $want" >&2; exit 1; }
|
||||
got="$(sha256sum "$path" | cut -d' ' -f1)"
|
||||
[ "$got" = "$sha" ] || { echo "SHA256 $path: $got != $sha" >&2; exit 1; }
|
||||
echo "OK $f ($want bytes, sha256)"
|
||||
done
|
||||
done
|
||||
echo "DOWNLOAD_DONE $DEST ($MODELS)"
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env bash
|
||||
# Fetch the refusal-direction adapter that frank-zealot applies at load:
|
||||
# AtomicChat/Ternary-Bonsai-2-9B-Uncensored-LoRA-GGUF — a **3.2 MB rank-1 LoRA**
|
||||
# for prism-ml/Ternary-Bonsai-2-9B, not a checkpoint. Public repo: no auth
|
||||
# header, and never a token on the command line (a Bearer token is
|
||||
# world-readable in /proc/<pid>/cmdline).
|
||||
#
|
||||
# IT IS AN EDIT INSIDE THE COMPUTE GRAPH, NOT A SECOND MODEL. The card: B = -r
|
||||
# (the refusal direction, row 23 of its own probe matrix), A = P^T(s *
|
||||
# H(r^T W_fold)) built from the base pack's folded input basis, applied at every
|
||||
# transformer layer (`target_layers = all`, `sites = 64`), with the embedding
|
||||
# deliberately untouched because an embedding LoRA crashes this fork's graph.
|
||||
# `--lora-scaled <file>:<scale>` is the flag pair: scale 1 is the card's
|
||||
# "held-out refusals 0 %" point, scale 0 is the plain model again.
|
||||
#
|
||||
# THE RUNTIME IS A GATE, NOT A PREFERENCE. The card verified the adapter on
|
||||
# `prism-b10777-e3de715` — the exact build frank-zealot runs — and says older
|
||||
# forks (e.g. `prism-b10709`) refuse these packs with `unsupported
|
||||
# prism.hadamard.version: 2`. The fork is already pinned for the payload's
|
||||
# custom tensor type 142; this adapter adds a second reason it cannot move.
|
||||
#
|
||||
# PROVENANCE ASYMMETRY, STATED HONESTLY. The GGUF's own metadata names the pack
|
||||
# the direction was fitted against:
|
||||
# prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf
|
||||
# prism.abliterate.base_sha256 = 3561df7ac7d26a71ffa991ac2db740b10841c3a350aa2676ccd3ff5d428b4e8b
|
||||
# frank-zealot serves the PQ2_0 + MTP + Q8_0-embedding packing (its sha256 is
|
||||
# c79fcdfa…, see download-frank-zealot-bonsai9.sh), which is NOT that file and is
|
||||
# not on this host. The adapter still applies — same tensor names, and the
|
||||
# metadata is provenance rather than a checksum gate — but its strength was
|
||||
# calibrated on a sibling quantization. What proves the effect here is the
|
||||
# behaviour of the endpoint (a refusal probe before and after the swap), not the
|
||||
# metadata and not the card's table, which was measured on PTQ1_0.
|
||||
#
|
||||
# Fresh downloads, no resume; -f/--fail-with-body so an HTTP error is not a
|
||||
# success, then byte count and sha256 (this repo is NOT LFS — the file is a
|
||||
# plain git object, so the sha below is the content hash of the Hub's bytes as
|
||||
# fetched, not an LFS oid). Idempotent: a complete file is skipped.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/download-frank-zealot-uncensored-lora.sh [DEST_DIR] [--verify-only]
|
||||
set -euo pipefail
|
||||
|
||||
DEST=""
|
||||
VERIFY_ONLY=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--verify-only) VERIFY_ONLY=1 ;;
|
||||
*) DEST="$arg" ;;
|
||||
esac
|
||||
done
|
||||
DEST="${DEST:-/home/dank/models/Bonsai2-9B-MTP}"
|
||||
|
||||
REPO="AtomicChat/Ternary-Bonsai-2-9B-Uncensored-LoRA-GGUF"
|
||||
# name | bytes | sha256 of the Hub's bytes
|
||||
FILES=(
|
||||
"Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf|3155424|861b9c099fb5cd81438ba85bad30590847c777b1e18112586be7f4f9ae5fb07b"
|
||||
)
|
||||
|
||||
mkdir -p "$DEST"
|
||||
status=0
|
||||
for entry in "${FILES[@]}"; do
|
||||
IFS='|' read -r name size sha <<<"$entry"
|
||||
url="https://huggingface.co/${REPO}/resolve/main/${name}"
|
||||
f="$DEST/$name"
|
||||
have=0
|
||||
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
|
||||
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
have=1
|
||||
fi
|
||||
fi
|
||||
if (( have )); then
|
||||
echo "OK $name"
|
||||
continue
|
||||
fi
|
||||
if (( VERIFY_ONLY )); then
|
||||
echo "MISSING $name"
|
||||
status=1
|
||||
continue
|
||||
fi
|
||||
echo "FETCHING $name (from $REPO)"
|
||||
tmp="$f.part"
|
||||
curl -fsSL --fail-with-body -o "$tmp" "$url"
|
||||
mv "$tmp" "$f"
|
||||
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
|
||||
echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
|
||||
status=1
|
||||
continue
|
||||
fi
|
||||
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
echo "SHA FAIL $name"
|
||||
status=1
|
||||
continue
|
||||
fi
|
||||
echo "VERIFIED $name"
|
||||
done
|
||||
if (( status == 0 )); then
|
||||
echo "DOWNLOAD_DONE $DEST"
|
||||
else
|
||||
echo "DOWNLOAD_FAILED $DEST" >&2
|
||||
fi
|
||||
exit "$status"
|
||||
@@ -80,3 +80,48 @@ HALOGEN_TOP_K=20
|
||||
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
||||
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
||||
HALOGEN_WEIGHTS_LOCK=1
|
||||
|
||||
# --- NPU lane (XDNA2) -------------------------------------------------------
|
||||
# HALOGEN_NPU_MODELS is the whole switch: EMPTY or unset = the endpoint exactly
|
||||
# as it ran before this lane existed (no accel device, no XRT mount, no
|
||||
# fabric-clock dependency, no /v1/embeddings). Rolling the lane back is blanking
|
||||
# this one line and re-deploying — nothing else moves.
|
||||
# Names are the vendor catalog's folder names (`/opt/halogen/npu/models.txt`
|
||||
# inside the image) and the engine reads a model ONLY at /models/npu/<name>/ —
|
||||
# HALOGEN_NPU_DIR is gone from the image and setting it is a boot refusal.
|
||||
# The engine cannot fetch anything here: /models is read-only and
|
||||
# HALOGEN_DOWNLOAD stays unset so the container opens no outbound connection at
|
||||
# boot, so scripts/download-frank-dragoon-npu.sh must stage the files first
|
||||
# (the serve gate and the deployer both refuse by name when a named model is
|
||||
# missing). FRANK_DRAGOON_NPU_DIR therefore lives INSIDE the payload dir: the
|
||||
# payload mount is the container's only handle on /models, and runc cannot
|
||||
# mkdir a nested /models/npu mountpoint through a read-only bind — the lane's
|
||||
# first boot died on precisely that (2026-10-19). Swap the payload dir and this
|
||||
# path moves with it.
|
||||
# Cost: ~1.5 GiB extra pinned host RAM per named model, and a hard dependency on
|
||||
# the host holding the GPU fabric clock — on Strix Halo the iGPU and the NPU
|
||||
# together hang the machine (and corrupt NPU results) while FCLK changes speed,
|
||||
# so halogen refuses to start with this variable set unless the clock is held
|
||||
# (dankclaw/host/halogen-fabric-clock; the deployer installs and gates it).
|
||||
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
||||
FRANK_DRAGOON_NPU_DIR=/home/dank/models/halogen-v2-abliterated/npu
|
||||
# Node paths and the XRT directory are variables so the bats seam can drive
|
||||
# these gates on a box with no NPU. Ubuntu's distro XRT (libxrt2 + libxrt-npu2)
|
||||
# is mounted FILE BY FILE and TWICE EACH — once where the image looks for XRT,
|
||||
# once at its own path, because those libraries find each other at their install
|
||||
# path; the vendor forbids mounting an /opt/xilinx/xrt of symlinks into the
|
||||
# system dir, which dangle inside the container.
|
||||
FRANK_DRAGOON_DEV_ACCEL=/dev/accel/accel0
|
||||
FRANK_DRAGOON_XRT_DIR=/usr/lib/x86_64-linux-gnu
|
||||
# sysfs root the fabric-clock preflight reads (same knob the vendor's
|
||||
# halogen-fabric-clock status reads; the seam points it at a fixture).
|
||||
FRANK_DRAGOON_SYSFS=/sys
|
||||
# Container network. Empty = podman's rootless default (`pasta`): a private
|
||||
# netns that nothing else can join — podman refuses a second network while the
|
||||
# primary is pasta, the published port is refused from dankclaw_default
|
||||
# containers, and host.containers.internal resolves link-local, which
|
||||
# open-notebook's URL guard refuses to store. Named = the app bridge, so
|
||||
# open-notebook can dial http://frank-dragoon-halogen:8731 for the NPU
|
||||
# endpoints. The engine's unauthenticated 8730 stays private either way (it
|
||||
# binds the container's own loopback); only the API port becomes name-resolvable.
|
||||
FRANK_DRAGOON_NETWORK=dankclaw_default
|
||||
|
||||
@@ -27,6 +27,16 @@
|
||||
# below stop describing the process you are running. Nothing in this fleet
|
||||
# pins this container; that is a decision, not an oversight.
|
||||
#
|
||||
# THE DEVICE LAW (since the 2026-10-19 Vulkan cutover). Synthesis computes on
|
||||
# the gfx1151 iGPU — the SAME card frank-dragoon serves. It is a shared device,
|
||||
# never a dedicated one, so two numbers matter and neither is what `rocm-smi`
|
||||
# prints first: the memory the container may use is the RADV heap inside the
|
||||
# GTT-backed aperture (the ~2 GiB DRM carve-out is not the working set), and the
|
||||
# timing that matters is dragoon's, measured WITH a synthesis in flight. The
|
||||
# rollback is one line: set QWENTTS_DEVICE empty, and the entrypoint passes no
|
||||
# --device at all — the pre-iGPU CPU shape, no rebuild, no restart of anything
|
||||
# else on the box.
|
||||
#
|
||||
# THE REGISTRY LAW (why the unit has an ExecStartPost at all). Registrations live
|
||||
# in memory only, so a restart loses them, and re-registration alone does NOT warm
|
||||
# a voice — the warm pass has to synthesize something. Without it the façade
|
||||
@@ -103,6 +113,17 @@ QWENTTS_LANG=auto
|
||||
# round-trip outside it) is what absorbs the queue.
|
||||
QWENTTS_MAX_BATCH=1
|
||||
|
||||
# The compute device, by ggml device NAME. `Vulkan0` is the gfx1151 iGPU: the
|
||||
# image carries the Mesa (RADV) ICD and no NVIDIA ICD, so this container's
|
||||
# Vulkan enumeration has exactly one entry and it is the iGPU. The unit hands
|
||||
# the container /dev/dri and keeps the render group; on this hybrid host the
|
||||
# iGPU's render node is renderD129 and renderD128 is the NVIDIA card, which is
|
||||
# why the unit passes the directory and lets the driver resolve the node.
|
||||
#
|
||||
# ROLLBACK: an empty value means no --device flag at all — the CPU shape this
|
||||
# endpoint shipped with until 2026-10-19. One line, no rebuild.
|
||||
QWENTTS_DEVICE=Vulkan0
|
||||
|
||||
# --- the boot warmer (dankclaw/voiceservice/tts-engine-warm.sh) --------------
|
||||
# These three are read by the warm hook, not by tts-server. They live here
|
||||
# because the hook is exec-ed into this container and therefore inherits this
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
|
||||
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file.
|
||||
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
|
||||
# the refusal-direction adapter applied at load.
|
||||
#
|
||||
# ONE OWNER: shape, payload identity, eras, laws and every measured number
|
||||
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
|
||||
@@ -13,6 +14,17 @@ MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.g
|
||||
# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
|
||||
# under concurrent image requests, which is exactly --parallel 3.
|
||||
MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
|
||||
# Refusal-direction adapter (a rank-1 LoRA, 3.2 MB) applied at load; it edits
|
||||
# the compute graph, it is not a second model. Scale 1 is the card's "held-out
|
||||
# refusals 0 %" point. LORA_FILE="" is the documented opt-out — the plain pack
|
||||
# again, no rebuild — and LORA_SCALE=0 is the same result through the flag.
|
||||
# PROVENANCE ASYMMETRY: the adapter's own prism.abliterate.base_gguf names
|
||||
# Ternary-Bonsai-2-9B-PTQ1_0.gguf as the pack its direction was fitted against,
|
||||
# which is NOT this packing and is not on this host. The tensor names match, so
|
||||
# it applies; the strength was calibrated on a sibling quantization, so the
|
||||
# effect is proven by the endpoint's behaviour, never by the metadata.
|
||||
LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf
|
||||
LORA_SCALE=1
|
||||
# Served id — the harness/family name, not a fact about the payload.
|
||||
# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
|
||||
ALIAS=frank-zealot
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[Unit]
|
||||
Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 2x262144 slots over one 524288 pool, unquantized f16 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
|
||||
Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF + refusal adapter (qwen35 hybrid, 3x200704 slots over one 602112 pool, f16 K + q8_0 V, MTP armed, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
|
||||
Executable
+211
@@ -0,0 +1,211 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Regenerate serverAppConfigs-docs/index.md from page frontmatter.
|
||||
|
||||
The root index is a locator (AGENTS.md -> *Documentation Policy*): one row per
|
||||
page, one clause, no numbers. Hand-maintaining it is how rows go missing, and
|
||||
the docs seam fails when a page is not linked from it, so it is generated:
|
||||
|
||||
* preamble everything above the first `## ` in the existing file is kept
|
||||
verbatim (that is the curated statement of the vault's law)
|
||||
* section from the page's directory, overridden by its frontmatter tags
|
||||
(`endpoint` / `retired-endpoint` split the LLM pages)
|
||||
* row text the page's frontmatter `description`, verbatim
|
||||
|
||||
A page with no description is a hard error rather than an empty cell, so the
|
||||
generator cannot silently produce a silent index. Sections with no pages are
|
||||
skipped. Deterministic: the same vault always yields the same bytes.
|
||||
|
||||
Sub-directory indices are hand-grouped (they carry curated prose), so they are
|
||||
synced rather than regenerated: dead rows dropped, row text refreshed from the
|
||||
page's description, uncovered siblings appended.
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
RESERVED = {"index.md", "log.md"}
|
||||
SECTIONS = [
|
||||
("Inference endpoints", "endpoint"),
|
||||
("Retired endpoints", "retired-endpoint"),
|
||||
]
|
||||
DIRECTORY_SECTIONS = [
|
||||
("Dankclaw", "Dankclaw services"),
|
||||
("Dankserver", "Dankserver services"),
|
||||
("FrankBook-Pro", "FrankBook-Pro"),
|
||||
("Desktop", "Workstations"),
|
||||
("Infrastructure", "Cross-host infrastructure"),
|
||||
("Architecture", "Repo architecture and policy"),
|
||||
("Operations", "Operations and incidents"),
|
||||
("Agent", "Agent / harness"),
|
||||
("Research", "Research and methodology"),
|
||||
]
|
||||
PIPELINE_SECTION = "Pipeline Parallelism (series)"
|
||||
LLM_FALLBACK = "LLM technique and hardware references"
|
||||
CHANGE_SECTION = "Change history"
|
||||
ORDER = [t for t, _ in SECTIONS] + [t for _, t in DIRECTORY_SECTIONS] + \
|
||||
[PIPELINE_SECTION, LLM_FALLBACK, CHANGE_SECTION]
|
||||
|
||||
|
||||
def frontmatter_fields(text):
|
||||
if not text.startswith("---"):
|
||||
return {}
|
||||
end = text.find("\n---", 3)
|
||||
if end == -1:
|
||||
return {}
|
||||
raw = text[3:end]
|
||||
fields = {}
|
||||
for key in ("description", "type"):
|
||||
m = re.search(rf"^{key}:\s*(.*)$", raw, re.M)
|
||||
if m:
|
||||
fields[key] = m.group(1).strip().strip("\"'")
|
||||
tags = []
|
||||
m = re.search(r"^tags:\s*\[(.*?)\]$", raw, re.M)
|
||||
if m:
|
||||
tags = [t.strip().strip("\"'") for t in m.group(1).split(",") if t.strip()]
|
||||
elif re.search(r"^tags:\s*$", raw, re.M):
|
||||
block = re.search(r"^tags:\s*\n((?:\s+-\s+.*\n?)+)", raw, re.M)
|
||||
if block:
|
||||
tags = [l.strip().lstrip("-").strip().strip("\"'")
|
||||
for l in block.group(1).splitlines() if l.strip()]
|
||||
fields["tags"] = tags
|
||||
return fields
|
||||
|
||||
|
||||
def section_for(page, vault, fields):
|
||||
rel = page.relative_to(vault)
|
||||
if len(rel.parts) == 1:
|
||||
return CHANGE_SECTION if page.name == "log.md" else "Uncategorised"
|
||||
top = rel.parts[0]
|
||||
if top == "LLM":
|
||||
if PIPELINE_SECTION_DIR in rel.parts:
|
||||
return PIPELINE_SECTION
|
||||
for title, tag in SECTIONS:
|
||||
if tag in fields.get("tags", []):
|
||||
return title
|
||||
return LLM_FALLBACK
|
||||
for prefix, title in DIRECTORY_SECTIONS:
|
||||
if top == prefix:
|
||||
return title
|
||||
return "Uncategorised"
|
||||
|
||||
|
||||
ROW = re.compile(r"^(\| )\[\[([^\]|]+)\]\]( \|)( .*?)( \|$)")
|
||||
BULLET = re.compile(r"^(- )\[\[([^\]]+)\]\]( — )(.*?)$")
|
||||
|
||||
|
||||
def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
|
||||
"""Keep a hand-grouped sub-index honest without erasing its grouping.
|
||||
|
||||
Rows are locators, so their text is the linked page's own description; a
|
||||
row whose page is gone is a dead promise and is dropped; a direct sibling
|
||||
with no row is appended under `## Pages` (that is what the link-integrity
|
||||
seam checks). Curated headings and prose are untouched.
|
||||
"""
|
||||
here = index.parent
|
||||
known = {p.stem for p in here.rglob("*.md")
|
||||
if ".AGENTTRASH" not in str(p) and p.name != "index.md"}
|
||||
lines = index.read_text().split("\n")
|
||||
out, dropped, refreshed = [], 0, 0
|
||||
for line in lines:
|
||||
# A row whose first cell is empty is a description with no page: the link
|
||||
# was deleted when the page was destroyed, so the row is a dead promise.
|
||||
if re.match(r"^\|\s*\|", line.strip()):
|
||||
dropped += 1
|
||||
continue
|
||||
for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
|
||||
m = pat.match(line)
|
||||
if not m:
|
||||
continue
|
||||
stem = m.group(2).strip()
|
||||
if stem not in known:
|
||||
dropped += 1
|
||||
line = None
|
||||
break
|
||||
cand = next((p for p in here.rglob(f"{stem}.md")
|
||||
if ".AGENTTRASH" not in str(p)), None)
|
||||
desc = frontmatter_fields(cand.read_text(errors="replace")).get("description") \
|
||||
if cand else None
|
||||
if desc:
|
||||
current = m.group(4).strip()
|
||||
if current != desc:
|
||||
line = wrap.format(stem, " " if pat is ROW else "", desc) \
|
||||
if pat is ROW else wrap.format(stem, desc)
|
||||
refreshed += 1
|
||||
break
|
||||
if line is not None:
|
||||
out.append(line)
|
||||
text = "\n".join(out)
|
||||
# Presence must be the wikilink itself: a bare stem is a substring of other
|
||||
# page names (`frank-medium` lives inside `frank-medium-rocmfpx`), and a
|
||||
# sibling matched that way silently never gets a row.
|
||||
missing = sorted(
|
||||
(p.stem, frontmatter_fields(p.read_text(errors="replace")).get("description", ""))
|
||||
for p in here.glob("*.md")
|
||||
if p.name != "index.md" and f"[[{p.stem}]]" not in text
|
||||
)
|
||||
if missing:
|
||||
block = ["", "## Pages", "", "| Page | Owns |", "|---|---|"]
|
||||
block += [f"| [[{stem}]] | {desc} |" for stem, desc in missing]
|
||||
text = text.rstrip("\n") + "\n" + "\n".join(block) + "\n"
|
||||
if text != index.read_text():
|
||||
index.write_text(text)
|
||||
return dropped, refreshed, len(missing)
|
||||
|
||||
|
||||
PIPELINE_SECTION_DIR = "Pipeline Parallelism"
|
||||
|
||||
|
||||
def main(argv):
|
||||
vault = pathlib.Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
|
||||
index = vault / "index.md"
|
||||
if not index.is_file():
|
||||
print(f"missing {index}")
|
||||
return 2
|
||||
existing = index.read_text()
|
||||
preamble = existing.split("\n## ", 1)[0].rstrip("\n")
|
||||
|
||||
pages = [p for p in vault.rglob("*.md")
|
||||
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
|
||||
and p.name != "index.md" and p != index]
|
||||
buckets, errors = {}, []
|
||||
for page in pages:
|
||||
fields = frontmatter_fields(page.read_text(errors="replace"))
|
||||
if page.name != "log.md":
|
||||
desc = fields.get("description")
|
||||
if not desc:
|
||||
errors.append(str(page.relative_to(vault)))
|
||||
continue
|
||||
else:
|
||||
desc = "Chronological incident and law log for the whole bundle."
|
||||
buckets.setdefault(section_for(page, vault, fields), []).append((page.stem, desc))
|
||||
if errors:
|
||||
print("pages with no description (fix the frontmatter, not the index):")
|
||||
for e in sorted(errors):
|
||||
print(" " + e)
|
||||
return 1
|
||||
|
||||
out = [preamble, ""]
|
||||
for title in ORDER + sorted(set(buckets) - set(ORDER)):
|
||||
rows = sorted(buckets.get(title, []), key=lambda r: r[0].lower())
|
||||
if not rows:
|
||||
continue
|
||||
out += [f"## {title}", "", "| Page | Owns |", "|---|---|"]
|
||||
out += [f"| [[{stem}]] | {desc} |" for stem, desc in rows]
|
||||
out.append("")
|
||||
out.append("<!-- generated by scripts/gen-vault-index.py; edit page frontmatter instead -->")
|
||||
index.write_text("\n".join(out) + "\n")
|
||||
touched = 0
|
||||
for sub in sorted(vault.rglob("index.md")):
|
||||
if sub == index or ".AGENTTRASH" in str(sub):
|
||||
continue
|
||||
dropped, refreshed, added = sync_subindex(sub, vault)
|
||||
if dropped or refreshed or added:
|
||||
touched += 1
|
||||
print(f" {sub}: -{dropped} dead rows, {refreshed} refreshed, +{added} added")
|
||||
print(f"{index}: {sum(len(v) for v in buckets.values())} rows in "
|
||||
f"{len([t for t in ORDER if buckets.get(t)])} sections, {touched} sub-indices synced")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
+27
-17
@@ -1,5 +1,6 @@
|
||||
# open-notebook.env — configuration for **open-notebook**, the fleet's podcast
|
||||
# generator on Dankclaw (UI host port **8502**, API host port **5056**). Read by
|
||||
# generator on Dankclaw (user path https://notebook.franksdank.cloud; UI host
|
||||
# port **8502**, API host port **5056 loopback-only**). Read by
|
||||
# BOTH Quadlet units (dankclaw/open-notebook/quadlet/) as the FIRST
|
||||
# EnvironmentFile= line, which is why every path and host name below is the
|
||||
# value the CONTAINER must see.
|
||||
@@ -47,13 +48,17 @@
|
||||
#
|
||||
# THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept
|
||||
# visible beside the TTS number because both belong to one podcast run: the
|
||||
# outline call and the transcript call are the slow text work, and frank-zealot
|
||||
# answers them well inside 180. Provider choice is made in the UI, not here —
|
||||
# see [[Open Notebook]] for the one-time credential provisioning — and it is
|
||||
# frank-zealot on purpose: script writing asks for a JSON-schema response, which
|
||||
# the OpenAI-compatible llama.cpp endpoint is known to serve, while halogen's
|
||||
# structured-output behaviour on frank-dragoon is UNVERIFIED. halogen remains
|
||||
# the unproven alternative, not the fallback.
|
||||
# outline call and the transcript call are the slow text work, and the endpoint
|
||||
# provisioned today answers them well inside 180. The provider itself is chosen
|
||||
# in the database, not here — [[Open Notebook]] owns that provisioning and the
|
||||
# address census behind it. Short version: frank-zealot was the plan (script
|
||||
# writing asks for a JSON-schema response, which llama.cpp serves and halogen's
|
||||
# behaviour on frank-dragoon is UNVERIFIED), but no Dankclaw LLM is dialable from
|
||||
# this container's netns, and the single address that reaches one —
|
||||
# `host.containers.internal`, 169.254.1.2 — the app refuses to store at all,
|
||||
# because link-local is a blocked address class. The writer is therefore
|
||||
# frank-archon (`http://192.168.0.235:7770/v1`), the only fleet LLM that both
|
||||
# answers from inside this netns and passes that check.
|
||||
#
|
||||
# THE DATABASE COORDINATES. SURREAL_URL addresses the sibling container BY NAME
|
||||
# on systemd-open-notebook (container DNS is scoped to the owning network, so
|
||||
@@ -73,15 +78,20 @@ SURREAL_PASS=
|
||||
OPEN_NOTEBOOK_ENCRYPTION_KEY=
|
||||
OPEN_NOTEBOOK_PASSWORD=
|
||||
|
||||
# WHERE THE BROWSER FINDS THE API. The UI auto-detects `<host-you-opened>:5055`,
|
||||
# and 5055 on this host belongs to seerr, so the API leaves the container on 5056
|
||||
# and this line has to say so. Both origins are listed in CORS_ORIGINS because
|
||||
# the API is called cross-origin from the UI's own port: the LAN origin is the
|
||||
# normal one, the localhost one covers a browser opened on Dankclaw itself. The
|
||||
# app logs a warning at start while CORS_ORIGINS is unset and defaults to `*`,
|
||||
# which is a wildcard in front of an endpoint that serves every episode's audio.
|
||||
API_URL=http://192.168.0.143:5056
|
||||
CORS_ORIGINS=http://192.168.0.143:8502,http://localhost:8502
|
||||
# WHERE THE BROWSER FINDS THE API. The guarded vhost
|
||||
# (notebook.franksdank.cloud → Caddy → 8502) is the only user path, and the
|
||||
# image's own Next.js layer rewrites `/api/*` to the FastAPI running beside it,
|
||||
# so ONE origin serves UI and API and API_URL has to name that vhost. This is
|
||||
# not cosmetic: unset, the app auto-detects `<host-you-opened>:5055` — the port
|
||||
# upstream ships, and the one seerr holds on this host — and hands the browser
|
||||
# an address nothing answers; pointed at the LAN `:5056` instead, every call
|
||||
# leaves the guarded origin and an authenticated browser session does not ride
|
||||
# it. Same-origin traffic needs no CORS at all, but CORS_ORIGINS stays set to
|
||||
# the one origin because the app warns at start while it is unset and defaults
|
||||
# to `*` — a wildcard in front of an endpoint that serves every episode's
|
||||
# audio. No trailing slash: the UI concatenates `${API_URL}/api/...`.
|
||||
API_URL=https://notebook.franksdank.cloud
|
||||
CORS_ORIGINS=https://notebook.franksdank.cloud
|
||||
|
||||
# Concurrency: see "THE CPU-SHUTDOWN GUARD" above. Defaults are 5 and 5.
|
||||
TTS_BATCH_SIZE=1
|
||||
|
||||
Executable
+100
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Port census: prove a merge/collapse lost no fact.
|
||||
|
||||
For every line that existed in the OLD set and not in the NEW set, check that
|
||||
each of its significant tokens (numbers, long identifiers, paths) still occurs
|
||||
somewhere in the NEW corpus. A line whose every token survives is a reworded
|
||||
line; a line with an unheld token is a candidate fact loss and gets printed.
|
||||
|
||||
port-census.py <new-glob-dir> <old-file> [<old-file> ...]
|
||||
|
||||
Old files may be `git-rev:path` (read from git) or a path (e.g. a trash copy).
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def toks(line):
|
||||
out = []
|
||||
for n in re.findall(r"\d[\d,]*(?:\.\d+)?", line):
|
||||
if len(n.replace(",", "").split(".")[0]) >= 3 or (
|
||||
"." in n and len(n.split(".")[-1]) >= 2 and len(n.replace(",", "")) >= 4
|
||||
):
|
||||
out.append(n)
|
||||
# Identifiers, flags, paths and backticked spans are the facts worth
|
||||
# tracking; ordinary prose words are not — they reword on every rewrite.
|
||||
# A hyphen alone is not evidence of an identifier (`long-standing`), so a
|
||||
# dashed word counts only with two hyphens (`frank-tts-engine`) or with a
|
||||
# digit, dot, slash, colon or `--` inside it.
|
||||
for w in re.findall(r"[A-Za-z][\w./:-]{7,}", line):
|
||||
core = w.strip(".,;:!?\"')]")
|
||||
if (core.startswith("--") or core.isupper()
|
||||
or re.search(r"[\d_/:.]", core) or core.count("-") >= 2):
|
||||
out.append(core)
|
||||
out += [s.strip() for s in re.findall(r"`([^`]+)`", line) if len(s) > 3]
|
||||
return out
|
||||
|
||||
|
||||
def held(tok, corpus):
|
||||
t = tok.replace(",", "")
|
||||
alt = rf"\b{re.escape(t)}\b"
|
||||
for c in corpus:
|
||||
if tok in c or t in c:
|
||||
return True
|
||||
if re.search(alt, c.replace(",", "")):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def main(argv):
|
||||
new_dir = pathlib.Path(argv[1])
|
||||
|
||||
def corpus_excluding(exclude):
|
||||
"""Every vault page except the one under test.
|
||||
|
||||
The corpus is what must still hold a fact AFTER the file is gone, so a
|
||||
file that lives inside the scanned directory has to be removed from it —
|
||||
otherwise every token is "held" by the file itself and the census proves
|
||||
nothing (it reports 0 losses for every page, which is the bug this
|
||||
guards).
|
||||
"""
|
||||
out = []
|
||||
for p in new_dir.rglob("*.md"):
|
||||
s = str(p)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
if exclude is not None and p.resolve() == exclude:
|
||||
continue
|
||||
out.append(p.read_text(errors="replace"))
|
||||
return out
|
||||
|
||||
lost = checked = 0
|
||||
for spec in argv[2:]:
|
||||
if spec.startswith("git-"):
|
||||
ref, path = spec[4:].split(":", 1)
|
||||
old = subprocess.run(["git", "show", f"{ref}:{path}"],
|
||||
capture_output=True, text=True).stdout
|
||||
label = f"{ref}:{path}"
|
||||
exclude = None
|
||||
else:
|
||||
old = pathlib.Path(spec).read_text(errors="replace")
|
||||
label = spec
|
||||
exclude = pathlib.Path(spec).resolve()
|
||||
blob = "\n".join(corpus_excluding(exclude))
|
||||
for line in old.splitlines():
|
||||
s = line.strip().strip("->*|# ").strip()
|
||||
if len(s) < 30 or s.startswith("```"):
|
||||
continue
|
||||
checked += 1
|
||||
missing = [t for t in toks(s) if not held(t, [blob])]
|
||||
if missing:
|
||||
lost += 1
|
||||
print(f" {label}\n L: {s[:150]}\n unheld: {', '.join(missing[:6])}")
|
||||
print(f"{checked} old lines checked, {lost} with unheld tokens")
|
||||
return 1 if lost else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Executable
+56
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Repair the two artifacts the destruction passes introduced.
|
||||
|
||||
1. The link stripper could eat the space after a `- ` marker (`-`X``) and leave a
|
||||
dangling `see .` where a wikilink used to be.
|
||||
2. The Eras collapse could swallow a table's HEADER row (it looked like just
|
||||
another non-current row), leaving a separator with no header above it.
|
||||
|
||||
Usage: repair-damage.py [--apply]
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
VAULT = pathlib.Path("serverAppConfigs-docs")
|
||||
ERAS_HEADER = "| Window | Payload | Runtime | Shape | Replaced because | Rollback |"
|
||||
|
||||
|
||||
def main(argv):
|
||||
apply = "--apply" in argv
|
||||
marker = dangling = header = 0
|
||||
for p in sorted(VAULT.rglob("*.md")):
|
||||
s = str(p)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
lines = p.read_text(errors="replace").split("\n")
|
||||
out = []
|
||||
# Only a pointer whose target was deleted reads as `… see .` — never touch
|
||||
# a line merely because it wraps on a comma. (An earlier version of this
|
||||
# rule stripped the trailing comma off 660 wrapped prose lines, which
|
||||
# `scripts/restore-punctuation.py` then had to undo from HEAD.)
|
||||
for line in lines:
|
||||
new = line
|
||||
if re.match(r"^(\s*)[-*](?![-* ])[$`[(*_]", new):
|
||||
new = re.sub(r"^(\s*)[-*]", r"\1- ", new)
|
||||
marker += 1
|
||||
if re.search(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", new):
|
||||
new = re.sub(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", "", new)
|
||||
dangling += 1
|
||||
out.append(new)
|
||||
# A separator row whose line above is not a header gets its header back.
|
||||
fixed = []
|
||||
for i, line in enumerate(out):
|
||||
if re.match(r"^\|-+", line.strip()) and (not fixed or not fixed[-1].strip().startswith("|")):
|
||||
fixed.append(ERAS_HEADER)
|
||||
header += 1
|
||||
fixed.append(line)
|
||||
if apply and "".join(fixed) != "".join(lines):
|
||||
p.write_text("\n".join(fixed))
|
||||
print(f"{'APPLIED' if apply else 'dry run'}: marker spacing {marker}, "
|
||||
f"dangling pointers {dangling}, restored table headers {header}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Executable
+66
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Undo trailing-punctuation damage by restoring lines that match HEAD exactly
|
||||
except for a stripped trailing `,` / `;` / removed trailing word.
|
||||
|
||||
The repair pass over-reached: its "dangling pointer" cleanup also matched every
|
||||
wrapped prose line ending in a comma, and deleted the comma. A line whose text
|
||||
is otherwise byte-identical to its HEAD version, differing only by trailing
|
||||
punctuation, is not an edit — it is damage, so HEAD's line goes back.
|
||||
|
||||
Usage: restore-punctuation.py [--apply]
|
||||
"""
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
REPO = pathlib.Path(".")
|
||||
VAULT = REPO / "serverAppConfigs-docs"
|
||||
|
||||
|
||||
def head_text(rel):
|
||||
r = subprocess.run(["git", "show", f"HEAD:{rel}"], capture_output=True, text=True)
|
||||
return r.stdout if r.returncode == 0 else None
|
||||
|
||||
|
||||
def main(argv):
|
||||
apply = "--apply" in argv
|
||||
restored = 0
|
||||
pages = 0
|
||||
for p in sorted(VAULT.rglob("*.md")):
|
||||
s = str(p)
|
||||
if ".AGENTTRASH" in s or ".obsidian" in s:
|
||||
continue
|
||||
rel = s
|
||||
head = head_text(rel)
|
||||
if head is None:
|
||||
continue
|
||||
cur = p.read_text(errors="replace").split("\n")
|
||||
# Key on (length, prefix) of the HEAD line minus trailing punctuation so a
|
||||
# damaged line can find its original.
|
||||
index = {}
|
||||
for hl in head.split("\n"):
|
||||
h = hl.rstrip()
|
||||
if h.endswith((",", ";")) or True:
|
||||
key = (len(h.rstrip(",;. ")), h.rstrip(",;. ")[:60])
|
||||
index.setdefault(key, h)
|
||||
touched = 0
|
||||
for i, cl in enumerate(cur):
|
||||
c = cl.rstrip()
|
||||
if not c or c.endswith((".", ":", "|", "`", ")", "," , ";")):
|
||||
continue
|
||||
key = (len(c), c[:60])
|
||||
hl = index.get(key)
|
||||
if hl and hl != c and hl.startswith(c):
|
||||
cur[i] = hl
|
||||
touched += 1
|
||||
if touched:
|
||||
pages += 1
|
||||
restored += touched
|
||||
if apply:
|
||||
p.write_text("\n".join(cur))
|
||||
print(f"{'APPLIED' if apply else 'dry run'}: {restored} lines restored across {pages} pages")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -45,7 +45,7 @@ set +a
|
||||
: "${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
|
||||
: "${FRANK_ARCHON_MODEL_DIR:?FRANK_ARCHON_MODEL_DIR must name the .rad dir}"
|
||||
: "${FRANK_ARCHON_MODEL_FILE:?FRANK_ARCHON_MODEL_FILE must name the .rad file}"
|
||||
: "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the /data state dir}"
|
||||
: "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the host dir holding the rad-tune cache}"
|
||||
: "${FRANK_ARCHON_CONTAINER:?FRANK_ARCHON_CONTAINER must name the container}"
|
||||
: "${FRANK_ARCHON_SERVED_NAME:?FRANK_ARCHON_SERVED_NAME must name the alias}"
|
||||
: "${FRANK_ARCHON_PORT:?FRANK_ARCHON_PORT must be the port}"
|
||||
|
||||
@@ -27,6 +27,11 @@
|
||||
# the flash-server line reads it; it ships as a container; the root unit
|
||||
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
|
||||
# exactly like frank-archon's.
|
||||
# Since 2026-10-19 the container also carries the XDNA2 NPU lane
|
||||
# (HALOGEN_NPU_MODELS): the vendor's precompiled AIE programs for the models
|
||||
# named there run on the NPU inside THIS container and answer /v1/embeddings,
|
||||
# beside the Flash model, on the same port. Opt-in — the variable empty is the
|
||||
# pre-lane endpoint, byte for byte.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
@@ -115,6 +120,107 @@ if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
||||
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- NPU lane (XDNA2), opt-in via HALOGEN_NPU_MODELS ------------------------
|
||||
# Empty = the endpoint exactly as it ran before the lane existed: no device, no
|
||||
# extra mounts, no gate below. Set = three host preconditions, each refused BY
|
||||
# NAME (the 2026-10-01 law: the serve gate and the deployer gate must agree):
|
||||
# 1. the staged model files under $FRANK_DRAGOON_NPU_DIR — a path that MUST
|
||||
# live INSIDE $FRANK_DRAGOON_MODEL_DIR, because the container's only
|
||||
# handle on them is the read-only /models mount. A nested bind at
|
||||
# /models/npu cannot work: runc has to mkdir its mountpoint and the parent
|
||||
# bind is ro (`mkdirat .../merged/models/npu: read-only file system`,
|
||||
# measured 2026-10-19), and the image ships no /models at all for it to
|
||||
# land on. `$MODEL_DIR/npu/<id>` is visible at /models/npu/<id> through
|
||||
# the existing mount, which is also where the vendor's catalog expects it.
|
||||
# The engine re-checks size+sha256 against that catalog at boot; it cannot
|
||||
# fetch anything here because /models is read-only and HALOGEN_DOWNLOAD
|
||||
# stays unset — no outbound connection at boot),
|
||||
# 2. the distro XRT's three libraries, each mounted twice (where the image
|
||||
# looks for XRT, and at its own path — the libraries find each other there;
|
||||
# an /opt/xilinx/xrt of symlinks into the system dir dangles inside the
|
||||
# container, which the vendor calls out explicitly),
|
||||
# 3. the GPU fabric clock HELD: on Strix Halo the iGPU and the NPU co-running
|
||||
# hang the machine while FCLK changes speed, and the engine refuses to
|
||||
# start when it moves — it cannot read host sysfs from inside the
|
||||
# container, so the proof lives here, checked exactly the way the vendor's
|
||||
# `halogen-fabric-clock status` checks it.
|
||||
NPU_ARGS=()
|
||||
NPU_MODELS="${HALOGEN_NPU_MODELS:-}"
|
||||
if [ -n "$NPU_MODELS" ]; then
|
||||
DEV_ACCEL="${FRANK_DRAGOON_DEV_ACCEL:-/dev/accel/accel0}"
|
||||
NPU_DIR="${FRANK_DRAGOON_NPU_DIR:?HALOGEN_NPU_MODELS is set, so FRANK_DRAGOON_NPU_DIR must name the staged model dir}"
|
||||
XRT_DIR="${FRANK_DRAGOON_XRT_DIR:-/usr/lib/x86_64-linux-gnu}"
|
||||
SYSFS="${FRANK_DRAGOON_SYSFS:-/sys}"
|
||||
# The lane's files have exactly one door into the container: the read-only
|
||||
# payload mount. A staging dir OUTSIDE it would need a nested bind whose
|
||||
# mountpoint runc cannot create through that read-only parent, so refuse the
|
||||
# shape by name here rather than let runc die at container init (2026-10-19).
|
||||
case "$NPU_DIR/" in
|
||||
"$FRANK_DRAGOON_MODEL_DIR"/*) ;;
|
||||
*) echo "frank-dragoon: FRANK_DRAGOON_NPU_DIR=$NPU_DIR is outside the payload dir $FRANK_DRAGOON_MODEL_DIR — the lane's files reach the container only through the read-only /models mount; move the staging dir inside it" >&2
|
||||
exit 1 ;;
|
||||
esac
|
||||
if [ ! -e "$DEV_ACCEL" ]; then
|
||||
echo "frank-dragoon: $DEV_ACCEL missing — the NPU lane needs the amdxdna node (is iommu=pt still on the boot line?)" >&2
|
||||
exit 1
|
||||
fi
|
||||
for M in ${NPU_MODELS//,/ }; do
|
||||
if [ ! -d "$NPU_DIR/$M" ]; then
|
||||
echo "frank-dragoon: NPU model $M not staged in $NPU_DIR;" >&2
|
||||
echo " fetch it: bash scripts/download-frank-dragoon-npu.sh" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
for L in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
|
||||
if [ ! -e "$XRT_DIR/$L" ]; then
|
||||
echo "frank-dragoon: XRT NPU library $XRT_DIR/$L missing — the NPU lane cannot load (Ubuntu libxrt2 + libxrt-npu2)" >&2
|
||||
exit 1
|
||||
fi
|
||||
REAL="$(readlink -f "$XRT_DIR/$L")"
|
||||
NPU_ARGS+=(-v "$REAL:/opt/xilinx/xrt/lib/$L:ro" -v "$REAL:$XRT_DIR/$L:ro")
|
||||
done
|
||||
HELD=0
|
||||
for D in "$SYSFS"/class/drm/card*/device; do
|
||||
[ -e "$D/pp_dpm_fclk" ] || continue
|
||||
[ -e "$D/power_dpm_force_performance_level" ] || continue
|
||||
PERF="$(cat "$D/power_dpm_force_performance_level" 2>/dev/null || true)"
|
||||
TOP="$(grep . "$D/pp_dpm_fclk" | tail -n 1)"
|
||||
if [ "$PERF" = high ] || { [ "$PERF" = manual ] \
|
||||
&& [ "$(grep -c '\*' "$D/pp_dpm_fclk")" = 1 ] && [ "${TOP%\*}" != "$TOP" ]; }; then
|
||||
HELD=1
|
||||
else
|
||||
echo "frank-dragoon: fabric clock at $D is '$PERF', not held — the NPU lane refuses to start (GPU+NPU together need FCLK pinned)" >&2
|
||||
echo " hold it: sudo halogen-fabric-clock hold" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
if [ "$HELD" = 0 ]; then
|
||||
echo "frank-dragoon: no amdgpu device with a fabric-clock control under $SYSFS/class/drm — cannot prove FCLK is held" >&2
|
||||
exit 1
|
||||
fi
|
||||
NPU_ARGS+=(--device "$DEV_ACCEL" \
|
||||
-e HALOGEN_NPU_MODELS="$NPU_MODELS")
|
||||
echo "frank-dragoon: NPU lane armed: $NPU_MODELS on $DEV_ACCEL (fabric clock held)" >&2
|
||||
fi
|
||||
|
||||
# Container network. Empty = podman's rootless default (`pasta`), which is a
|
||||
# private netns and CANNOT be joined by anything else: podman refuses a second
|
||||
# network while the primary is pasta ("cannot set multiple networks without
|
||||
# bridge network mode"), the endpoint's published port answers 200 on the host
|
||||
# and on the LAN but is REFUSED from a dankclaw_default container (measured
|
||||
# 2026-10-19), and `host.containers.internal:7770` answers 200 yet resolves to
|
||||
# 169.254.1.2 — link-local, which open-notebook's URL guard refuses to store.
|
||||
# So an endpoint on pasta has no address an app container can use; naming the
|
||||
# app bridge here is the only shape that lets open-notebook dial
|
||||
# http://frank-dragoon-halogen:8731 for the NPU endpoints. It does not widen
|
||||
# the surface: the engine's unauthenticated 8730 is bound to the container's
|
||||
# OWN loopback (/proc/net/tcp: 127.0.0.1:8730 vs 0.0.0.0:8731, checked the same
|
||||
# day), so only the API port is reachable by name.
|
||||
NET_ARGS=()
|
||||
if [ -n "${FRANK_DRAGOON_NETWORK:-}" ]; then
|
||||
NET_ARGS=(--network "$FRANK_DRAGOON_NETWORK")
|
||||
fi
|
||||
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
||||
# The engine refuses to pin when MemAvailable is below the checkpoint plus
|
||||
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
|
||||
@@ -189,6 +295,8 @@ podman run --name "$FRANK_DRAGOON_CONTAINER" \
|
||||
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
|
||||
-v "$FRANK_DRAGOON_MODEL_DIR:/models:ro" \
|
||||
-v "$FRANK_DRAGOON_MODEL_DIR/tokenizer:/tokenizer:ro" \
|
||||
"${NET_ARGS[@]}" \
|
||||
"${NPU_ARGS[@]}" \
|
||||
"$FRANK_DRAGOON_IMAGE" \
|
||||
> "$LOG" 2>&1
|
||||
|
||||
|
||||
@@ -33,6 +33,19 @@
|
||||
# what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
|
||||
# opt-out.
|
||||
#
|
||||
# A refusal-direction adapter rides on top of the pack (Frank, 2026-10-19):
|
||||
# AtomicChat's rank-1 LoRA, applied at load by --lora-scaled. It is an edit
|
||||
# inside the compute graph, not a second model — B = -r (the refusal direction),
|
||||
# A built from the base pack's folded input basis, every transformer layer, the
|
||||
# embedding deliberately untouched because an embedding LoRA crashes this fork's
|
||||
# graph. The card verified it on prism-b10777-e3de715 — the build pinned here —
|
||||
# and says older forks refuse the pack outright (`unsupported
|
||||
# prism.hadamard.version: 2`), which makes the runtime lock doubly load-bearing.
|
||||
# Its own metadata names the PTQ1_0 packing as the pack the direction was fitted
|
||||
# against, so what proves the effect is a refusal probe against the live
|
||||
# endpoint, not the metadata and not the card's table. LORA_FILE="" drops the
|
||||
# flag and the endpoint is the plain pack again.
|
||||
#
|
||||
# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
|
||||
# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
|
||||
# what made it lose that evening was the bill, not the drafts — the draft context
|
||||
@@ -136,6 +149,13 @@ fi
|
||||
# Vision projector. No colon on purpose: an explicitly empty value means
|
||||
# "text-only, on purpose" and must NOT be replaced by the default path.
|
||||
: "${MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf}"
|
||||
# Refusal adapter. No colon on purpose, the same convention as MMPROJ_FILE: an
|
||||
# explicitly empty LORA_FILE is the documented opt-out (the plain pack again)
|
||||
# and must NOT be replaced by the default path. The scale travels INSIDE the
|
||||
# same argv token as the file, so LORA_SCALE only matters while LORA_FILE names
|
||||
# something; 0 is the card's "switch it off without touching the file" value.
|
||||
: "${LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf}"
|
||||
: "${LORA_SCALE:=1}"
|
||||
|
||||
# Fail loudly rather than half-loading: a missing file is a download that
|
||||
# has not finished, and Restart=always would turn it into a crash loop that
|
||||
@@ -149,6 +169,17 @@ if [[ -n "$MMPROJ_FILE" ]]; then
|
||||
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
||||
fi
|
||||
|
||||
# The adapter reaches the engine as ONE token — --lora-scaled <file>:<scale> —
|
||||
# so a scale can never arrive without its file. It is applied once, at load;
|
||||
# nothing here hot-swaps it (--lora-init-without-apply plus POST /lora-adapters
|
||||
# is that shape, and a process holding three conversations is not the place to
|
||||
# re-parameterise weights mid-flight). A missing file is a fetch that never
|
||||
# finished, so refuse rather than let Restart=always loop on it.
|
||||
if [[ -n "$LORA_FILE" ]]; then
|
||||
[[ -e "$LORA_FILE" ]] || { echo "frank-zealot: missing $LORA_FILE" >&2; exit 1; }
|
||||
EXTRA_ARGS+=(--lora-scaled "${LORA_FILE}:${LORA_SCALE}")
|
||||
fi
|
||||
|
||||
# Prometheus scraping is opt-IN in llama.cpp: without --metrics the server
|
||||
# answers 501 on /metrics, and the fleet scrape then reports the endpoint DOWN.
|
||||
if [[ "$METRICS" = "1" ]]; then
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Agent Docs and Edits
|
||||
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
|
||||
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
|
||||
tags: [agents, docs, markdown, hashline, edit-recovery]
|
||||
timestamp: 2026-10-18T00:00:00Z
|
||||
---
|
||||
@@ -21,7 +21,7 @@ Damage catalogue:
|
||||
no longer frontmatter (title/tags/description become body text).
|
||||
2. A rewrite **appended** instead of merged → duplicate headings and a stray
|
||||
second `---` mid-file.
|
||||
3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`,
|
||||
3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`
|
||||
`+- `, `- - `.
|
||||
4. Wrapped bullets re-emitted with a `- ` prefix on every physical line.
|
||||
5. Line-range replacement that dropped a sentence tail → dangling fragment,
|
||||
@@ -46,7 +46,7 @@ Repair order (matters):
|
||||
2. If it merely duplicated: **salvage unique content first** — cut the
|
||||
one-of-a-kind section, paste it into the copy you keep, THEN delete the
|
||||
duplicate ranges. Never delete first.
|
||||
3. Re-add missing sentence tails (insert before the orphaned fragment),
|
||||
3. Re-add missing sentence tails (insert before the orphaned fragment)
|
||||
keeping list continuation lines indented.
|
||||
4. Re-run all checks above.
|
||||
|
||||
@@ -57,7 +57,7 @@ safe, but its snapshot tag changes on every edit — re-read before each
|
||||
subsequent edit. `.AGENTTRASH/` is gitignored (`.gitignore:25`), so NEW trash
|
||||
stays local — but do not read that as "the repo has none of it": on 2026-10-18
|
||||
`git ls-files | grep -c AGENTTRASH` returns **37** tracked paths
|
||||
(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`,
|
||||
(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`
|
||||
`tests/.AGENTTRASH/`), because gitignore never untracks what was committed
|
||||
before the rule. Trash is a temporary hold, not an archive.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Command Center Dotfiles
|
||||
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
|
||||
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
|
||||
tags: [dotfiles, omp, deploy, rsync, blankpad]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: OMP Harness
|
||||
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
|
||||
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
|
||||
tags: [omp, harness, compaction, models-yml, context, samplers]
|
||||
timestamp: 2026-10-18T00:00:00Z
|
||||
---
|
||||
@@ -16,7 +16,7 @@ this file is repo-managed; edits are harness-side only.
|
||||
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
||||
since 2026-10-09):** `frank-archon` (Dankserver), `frank-dragoon` and
|
||||
`frank-zealot` (Dankclaw) — engine, port and card behind each id live on
|
||||
[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`,
|
||||
[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`
|
||||
`frank-zealot`. The SERVICES keep the old names (units, scripts, env
|
||||
files, runbook pages); pages written in the old era name the service,
|
||||
the harness names the model. The wire now matches: each endpoint's
|
||||
@@ -26,7 +26,7 @@ picker/grants, the homepage docs and the raw API all use one id per model
|
||||
note on the endpoint echoing the requested name).
|
||||
|
||||
Verified 2026-10-18: `/v1/models` on 192.168.0.235:7770, 192.168.0.143:7770 and
|
||||
192.168.0.143:7771 each answer with the unit name (`frank-archon`,
|
||||
192.168.0.143:7771 each answer with the unit name (`frank-archon`
|
||||
`frank-dragoon`, `frank-zealot`).
|
||||
|
||||
## Compaction chain arithmetic (verify before ANY window change)
|
||||
@@ -40,11 +40,11 @@ breaks compaction.
|
||||
Where the contract lives:
|
||||
|
||||
- `~/.omp/agent/config.yml` → `compaction.remoteEndpoint` (target URL:
|
||||
`http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`),
|
||||
`methodOrder` (`shake → snapcompact → soft → handoff → remote`,
|
||||
`http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`)
|
||||
`methodOrder` (`shake → snapcompact → soft → handoff → remote`
|
||||
`config.yml:77-82`), `thresholdPercent` (**95**, `config.yml:84`), idle
|
||||
settings (`idleEnabled`, `idleThresholdTokens: 100000`)
|
||||
- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`,
|
||||
- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`
|
||||
`compactionModel` references
|
||||
- Implementation: omp plugin
|
||||
`~/.omp/plugins/node_modules/@earendil-works/pi-coding-agent/dist/core/compaction/compaction.js`
|
||||
@@ -104,16 +104,16 @@ input. Boot-shape OOM laddering is a different problem
|
||||
## Per-model sampler config (`compat.extraBody`)
|
||||
|
||||
`models.yml` has **no per-model sampler keys**. The global knobs are
|
||||
`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`,
|
||||
`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`
|
||||
`repetitionPenalty`, and a knob reaches the wire only when it is set — the
|
||||
bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`),
|
||||
bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`)
|
||||
and this workstation's `config.yml` sets **none** of them. The 2026-10-05 note
|
||||
that the unset default is `-1`/"not sent" is retired: there is no `-1`
|
||||
sentinel, an unset key is simply absent. The per-model hook is
|
||||
`compat.extraBody`:
|
||||
|
||||
**Why it works:** `applyOpenAIExtraBody` runs **last** in the
|
||||
openai-completions builder — after `applyChatCompletionsCompatPolicy` — via
|
||||
openai-completions builder — after `applyChatCompletionsCompatPolicy` —
|
||||
`Object.assign(params, extraBody)`. So extraBody overrides global settings
|
||||
and compat-policy output.
|
||||
|
||||
@@ -191,7 +191,7 @@ assistant turn carrying a long `reasoning_content`, then compare
|
||||
`usage.prompt_tokens` with the flag true vs false. A large delta means the
|
||||
field changed template rendering; zero delta means it was ignored.
|
||||
|
||||
Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`),
|
||||
Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`)
|
||||
never `rm`.
|
||||
|
||||
## Static-preamble audit ("the context feels bloated")
|
||||
@@ -205,7 +205,7 @@ What loads statically every session (measure each, don't guess):
|
||||
```bash
|
||||
python3 - <<'EOF'
|
||||
import glob, re, os
|
||||
for root in [os.path.expanduser("~/.omp/agent/managed-skills"),
|
||||
for root in [os.path.expanduser("~/.omp/agent/managed-skills")
|
||||
os.path.expanduser("~/.omp/agent/skills"), ".omp/skills"]:
|
||||
n=tot=0
|
||||
for sk in glob.glob(root+"/*/SKILL.md"):
|
||||
@@ -305,4 +305,4 @@ standalone; fold go/no-go arithmetic skills into their probe skill
|
||||
- [[Agent Docs and Edits]] — editing vault docs and hashline targets safely
|
||||
- [[Command Center Dotfiles]] — the `~/.omp` repo deploy this config rides
|
||||
- [[Llama Cpp Serving Sizing]] — the engine-side sizing the chain must fit
|
||||
- [[LLM Bench Windows and Measurement]] — measuring the endpoints this config routes
|
||||
- measuring the endpoints this config routes
|
||||
|
||||
@@ -1,23 +1,12 @@
|
||||
---
|
||||
type: index
|
||||
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
|
||||
tags: [agent, index]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
# Agent
|
||||
|
||||
Pages about the agent's own tooling and context — how omp is configured, how
|
||||
docs get edited safely, and how the agent's home is deployed.
|
||||
|
||||
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
|
||||
`compat.extraBody`, preamble and skill-library audits
|
||||
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
|
||||
hashline edit recovery
|
||||
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
|
||||
transport, override packages
|
||||
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
|
||||
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
|
||||
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
|
||||
|
||||
## Related
|
||||
|
||||
- [[Testing Policy]] — the seams these pages' checks ride
|
||||
- [LLM](../LLM/index.md) — the fleet these agents operate
|
||||
- the fleet these agents operate
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Container Network Namespaces
|
||||
description: Which local services are addressable from which Podman network namespace on
|
||||
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
|
||||
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
|
||||
tags:
|
||||
- networking
|
||||
- podman
|
||||
@@ -48,7 +47,7 @@ network namespace**:
|
||||
|
||||
`Network=` values are read from the live Quadlet sources in
|
||||
`/home/dank/.config/containers/systemd/*.container`; runtime network names on the
|
||||
host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`,
|
||||
host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`
|
||||
`systemd-searxng`, `systemd-open-webui`.
|
||||
|
||||
Two consequences that are easy to miss:
|
||||
@@ -84,13 +83,49 @@ podman exec <name> cat /proc/net/route # default route → gateway IP
|
||||
podman exec <name> sh -c 'timeout 2 bash -c "exec 3<>/dev/tcp/10.89.1.1/8080"; echo rc=$?'
|
||||
```
|
||||
|
||||
Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`,
|
||||
Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`
|
||||
`10.89.1.1:3000` (Gitea, published), `:8081` (SearXNG via Gluetun's netns,
|
||||
published) and `:9119` (its own published port) connect; `:8080` and `:8787`
|
||||
are **refused instantly** even though both listeners are bound to `0.0.0.0` in
|
||||
the host netns. Probes to `host.containers.internal` (`169.254.1.2`) from that
|
||||
network time out rather than refuse — the alias is an address on the *default*
|
||||
`podman` network, and this container is not on it.
|
||||
network time out rather than refuse — but see the 2026-10-19 re-measure below,
|
||||
where the same alias answers from another `dankclaw_default` container.
|
||||
|
||||
### 2026-10-19: which published ports a rootless container can actually dial
|
||||
|
||||
Probed from inside `open-notebook` (rootless, on `dankclaw_default` **and** the
|
||||
private `systemd-open-notebook`), each address aimed at a live listener:
|
||||
|
||||
| Dial | Result | Mechanism |
|
||||
|---|---|---|
|
||||
| `192.168.0.143:3000` (Gitea) | **200** | published by a **rootless** container — rootless egress crosses the host netfilter where netavark installed that DNAT |
|
||||
| `192.168.0.143:7770` (`frank-dragoon`) | refused | the container is **rootless on podman's default `pasta` netns** (the root unit's `User=dank` runs `podman run`, so it lands in dank's storage). Pasta forwards the published port in the host netns — the host and the LAN both answer 200 — but installs no netavark DNAT for traffic arriving from a rootless bridge, so the same address is refused from a container. This is NOT a root/rootless rule-set split: `podman ps` as `dank` lists it |
|
||||
| `host.containers.internal:7770` (`169.254.1.2`) | **200** | the pasta forwarder is bound `0.0.0.0` on the host, so the gateway alias reaches it — and the alias is link-local, which is the dead end below |
|
||||
| `192.168.0.143:7771` (`frank-zealot`) | refused | host-netns listener, unpublished — the rule above, reconfirmed |
|
||||
| `10.89.1.1:7771` | refused | the bridge gateway address belongs to the rootlesskit netns, not to the host |
|
||||
| `host.containers.internal:7771` (`169.254.1.2`) | **200** | the gateway alias is the one address that carries a rootless container to a host-netns listener. Why `hermes-dashboard` timed out on this alias in 2026-09-02 while `open-notebook` answers today is **not established**; the difference to check first is the second network and where the default route lands |
|
||||
| `192.168.0.235:7770` (`frank-archon`) | **200** | a different machine: plain routing, no DNAT in the path |
|
||||
|
||||
The consequence that changes designs: **a rootless container's menu of local model
|
||||
endpoints is `frank-tts` (container DNS, same network) and `frank-archon` (LAN
|
||||
address on another host).** A `pasta` endpoint has no address an app container can
|
||||
use: the LAN address is refused, and the one alias that answers is link-local,
|
||||
which some applications refuse to store at all ([[Open Notebook]] meets exactly
|
||||
that refusal). Pasta cannot be *added to* either —
|
||||
|
||||
```
|
||||
podman run --network pasta --network dankclaw_default …
|
||||
Error: cannot set multiple networks without bridge network mode, selected mode pasta
|
||||
```
|
||||
|
||||
so the only shape that makes such an endpoint nameable is **moving it onto
|
||||
`dankclaw_default`** (`FRANK_DRAGOON_NETWORK` in `scripts/frank-dragoon.env`,
|
||||
2026-10-19). For frank-dragoon that is not an exposure change: the engine's
|
||||
unauthenticated token port binds the container's **own loopback**
|
||||
(`/proc/net/tcp` on 2026-10-19: `127.0.0.1:8730`, `0.0.0.0:8731`), so only the
|
||||
API port becomes name-resolvable, which is the same trust the published 7770
|
||||
already carries on the LAN. `frank-zealot` (7771) stays a host-netns binary and
|
||||
remains unreachable except through the link-local alias.
|
||||
|
||||
## Operational rules
|
||||
|
||||
@@ -99,9 +134,13 @@ network time out rather than refuse — the alias is an address on the *default*
|
||||
host — and never the reverse either: the deployer's health probe runs on the
|
||||
host, so it sees host-netns listeners that the container itself cannot dial.
|
||||
- To give a bridge-net container access to a host-netns listener, the options
|
||||
are (a) move it to `Network=host`, or (b) publish the needed port. Option (b)
|
||||
makes that port reachable from every netns and from the LAN — an exposure
|
||||
change that needs operator approval before deployment, not a silent fix.
|
||||
are (a) move it to `Network=host`, or (b) publish the needed port — but
|
||||
publishing is advertised **only inside the Podman that owns the container**:
|
||||
a root container's published port stays invisible to a rootless consumer
|
||||
(2026-10-19 table above), so the fix must be made on the same root/rootless
|
||||
side as the consumer. Either way it is an exposure change — a published port
|
||||
also answers on the LAN — and needs operator approval before deployment, not
|
||||
a silent fix.
|
||||
- `chrome-devtools-protocol` is intentionally `Network=host` with a loopback-only
|
||||
bind so the browser tool stays unreachable from the LAN; do not "tidy" it onto
|
||||
a bridge network — its two consumers (gateway, webui) are host-netns and reach
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Deploy State Triage
|
||||
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
|
||||
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
|
||||
tags: [deployment, deploy-state, systemd, podman, triage]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
@@ -36,7 +36,7 @@ possibly zero behavioral change.
|
||||
2. Probe the target (ssh as `dank`, passwordless sudo present):
|
||||
- `cat ~/.local/state/serverAppConfigs/deploy-state/<svc>`
|
||||
- `git -C ~/serverAppConfigs rev-parse HEAD`
|
||||
- `systemctl is-active/is-enabled <svc>`;
|
||||
- `systemctl is-active/is-enabled <svc>`
|
||||
`curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:<port>/health`
|
||||
- the on-host env file already carries the new keys (it landed via the
|
||||
ff-pull).
|
||||
@@ -119,8 +119,8 @@ each, kill stale ones by explicit PID.
|
||||
A killed deploy leaves behind:
|
||||
|
||||
- the unit **stopped/failed** (`Restart=always` does NOT fire after an
|
||||
explicit stop — start it yourself);
|
||||
- the state marker **pending** (pre-deploy or in-flight revision);
|
||||
explicit stop — start it yourself)
|
||||
- the state marker **pending** (pre-deploy or in-flight revision)
|
||||
- the e2e gate un-run (the deployer runs it last).
|
||||
|
||||
Reconcile the marker ONLY after the service itself is verified healthy — a
|
||||
@@ -142,11 +142,11 @@ Boot failures must be read from the journal, not `podman logs`: serve scripts
|
||||
numbers — which looks exactly like your change having no effect. Key on
|
||||
container identity (`podman ps -a --format '{{.CreatedAt}}'`) and read the
|
||||
cause from `journalctl --since "N min ago" | grep -a "<container-name>\["`.
|
||||
See [[vLLM Boot Forensics and Config Interrogation]] for the full capture
|
||||
See for the full capture
|
||||
procedure.
|
||||
|
||||
## Related
|
||||
|
||||
- [[Deployment Flow]] — the deploy pipeline itself
|
||||
- [[Quadlet Lifecycle]] — unit ownership and drift diagnosis
|
||||
- [[vLLM Boot Forensics and Config Interrogation]] — journal capture
|
||||
- journal capture
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
type: guide
|
||||
type: howto
|
||||
title: Deployment Flow
|
||||
description: Service-scoped Git deployment for the two homelab hosts.
|
||||
tags:
|
||||
@@ -68,7 +68,7 @@ service on the other host.
|
||||
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
||||
|
||||
The shared chat template maps to the inference service on the selected target.
|
||||
The `frank-npu` endpoint is retired ([[frank-npu]]) and has no deploy handler;
|
||||
The `frank-npu` endpoint is retired () and has no deploy handler
|
||||
an unrelated large-model deployment never touches it.
|
||||
|
||||
## Caddy safety
|
||||
@@ -125,6 +125,5 @@ routing. See [[Testing Policy]].
|
||||
## See Also
|
||||
|
||||
- [[Testing Policy]]
|
||||
- [LLM deployment](../LLM/deployment.md)
|
||||
- [[Caddy]]
|
||||
- [[SearXNG]]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
|
||||
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
|
||||
tags:
|
||||
- architecture
|
||||
- e2e
|
||||
@@ -49,7 +49,7 @@ user sees it; the `bats` seams assert config shape, not behavior.
|
||||
*arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a
|
||||
sleeping Mac mini must never fail an unrelated deploy. The project joins only
|
||||
when `E2E_INCLUDE_FRANKBOOK=1` (set by `scripts/deploy.sh` when a frankbook
|
||||
trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`,
|
||||
trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`
|
||||
`shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the arr
|
||||
vhosts) or when an explicit `--project` argument names it. An `11-admin` edit
|
||||
that touches the arr links must therefore be paired with a `19-frankbook` run
|
||||
|
||||
@@ -15,7 +15,7 @@ One Caddy fronts both hosts. It runs as a container on Dankclaw (192.168.0.143)
|
||||
|
||||
As of 2026-08-28 the front door is a two-tier chain: Caddy terminates TLS,
|
||||
Anubis verifies the client (DENY known AI scrapers, CHALLENGE everyone
|
||||
else), and accepted traffic returns to Caddy's internal listener on `:8016`,
|
||||
else), and accepted traffic returns to Caddy's internal listener on `:8016`
|
||||
which proxies every request whole to the Authentik server with the Host
|
||||
header preserved; Authentik's embedded outpost routes on that Host against
|
||||
each proxy provider's `external_host`, serves the login flow in place for
|
||||
@@ -67,12 +67,12 @@ graph TB
|
||||
```
|
||||
|
||||
Guarded (Anubis `:8923` → Caddy `:8016` → Authentik outpost → Caddy `:8017`):
|
||||
homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`),
|
||||
homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`)
|
||||
prometheus (`:135-137`), loki (`:141-143`), seerr (`:148-150`), marreta
|
||||
(`:155-160`), romm (`:167-169`), files (`:172-174`). Exempt / own-auth: `auth.`
|
||||
(the IdP login itself), `ai.` (OpenWebUI, its own OIDC), gitea (git smart-HTTP
|
||||
clients cannot solve JS challenges — `/info/refs` and `git-*-pack` pass
|
||||
unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP),
|
||||
unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP)
|
||||
books/jellyfrank (own logins), `hermes.`/`hermesadmin.`, and the *arr vhosts
|
||||
that terminate on FrankBook-Pro (`:254-281`, "exempt by construction: own
|
||||
auth"). Grafana is **not** exempt even though it signs in through authentik as
|
||||
@@ -104,13 +104,13 @@ Caddy holds one Let's Encrypt certificate per vhost, issued and renewed through
|
||||
`Network=searxng.network` (runtime network `systemd-searxng`), while
|
||||
`searxng.container`, `marreta.container` and `marreta-browser.container` join
|
||||
the tunnel the modern way — `Network=gluetun.container`
|
||||
(`dankclaw/searxng/quadlet/searxng.container:18`,
|
||||
`dankclaw/marreta/quadlet/marreta.container:34`,
|
||||
(`dankclaw/searxng/quadlet/searxng.container:18`
|
||||
`dankclaw/marreta/quadlet/marreta.container:34`
|
||||
`dankclaw/marreta/quadlet/marreta-browser.container:25`), the Quadlet
|
||||
equivalent of the retired compose `network_mode: service:gluetun`. **Five**
|
||||
containers share the VPN exit, not three, and Marreta's outbound fetches leave
|
||||
through the same WireGuard tunnel as SearXNG's. Pi-hole is a different kind of
|
||||
exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`);
|
||||
exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`)
|
||||
everything else on Dankclaw uses the normal LAN route. This matches KB Pattern
|
||||
2 in [[Gluetun]].
|
||||
See [[Container Network Namespaces]] for which addresses are reachable from that
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Quadlet Lifecycle
|
||||
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
|
||||
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
|
||||
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
@@ -36,14 +36,14 @@ done
|
||||
## 2. Cutover: compose service → Quadlet unit
|
||||
|
||||
Checklist (precedents to copy:
|
||||
`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`,
|
||||
`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`
|
||||
`dankclaw/observability/quadlet/`):
|
||||
|
||||
1. Unit file at `<host>/<svc>/quadlet/<name>.container`, mirroring compose
|
||||
line for line: `Image=` (`localhost/` prefix for locally built images +
|
||||
`Pull=never`), `ContainerName=` (Caddy and scrape configs address
|
||||
containers by name), `Network=`, `User=0:0` when the image's own user
|
||||
cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved),
|
||||
cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved)
|
||||
`Exec=` for compose `command:`, `PublishPort=`, health keys,
|
||||
`[Service] Restart=always` + `RestartSec=10`, `[Install]
|
||||
WantedBy=default.target`. **Never `systemctl --user enable` the generated
|
||||
@@ -56,7 +56,7 @@ Checklist (precedents to copy:
|
||||
`systemctl --user restart <name>.service` → `wait_for_health`; retire any
|
||||
legacy hand-written unit into `~/.AGENTTRASH/` (a leftover
|
||||
`~/.config/systemd/user/<svc>.service` that shells out to
|
||||
`podman compose up` collides BY NAME with the generated unit);
|
||||
`podman compose up` collides BY NAME with the generated unit)
|
||||
`podman rm -f <container>` so the unit's container can take the name.
|
||||
4. Planner: map `<host>/<svc>/**` → service in `scripts/deploy-plan.py`
|
||||
AND add a `service_is_allowed` entry; keep `SERVICE_ORDER` dependencies
|
||||
@@ -123,7 +123,7 @@ the restart policy faithfully resurrects it forever.
|
||||
|
||||
Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
|
||||
= the §2 cutover. **Verify the capability, not the status:** `podman exec
|
||||
<name> python3 -c "import torch; print(torch.cuda.is_available())"`;
|
||||
<name> python3 -c "import torch; print(torch.cuda.is_available())"`
|
||||
`ss -ltn | grep <port>` for the actual bind addresses. ComfyUI incident:
|
||||
2026-09-16, stale device-less container, 9,146 restarts. History repeats:
|
||||
2026-08-26 Caddy (no unit), 2026-09-11 five observability services.
|
||||
@@ -143,7 +143,7 @@ Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
|
||||
unhealthy but logs look fine, compare the baked
|
||||
`{{.Config.Healthcheck.Test}}` array against the intended command before
|
||||
believing the status.
|
||||
- **Probe a new key before shipping:** drop a throwaway `probe.container`,
|
||||
- **Probe a new key before shipping:** drop a throwaway `probe.container`
|
||||
`daemon-reload`, `journalctl --user --since "1 min ago" -o cat | grep -i
|
||||
"unsupported key"`, `systemctl --user cat probe.service` (shows the
|
||||
generated ExecStart verbatim), then remove + reload.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Stale Signal Audit
|
||||
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
|
||||
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
|
||||
tags: [testing, observability, audit, signals, gates]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
|
||||
@@ -54,4 +54,3 @@ Tests are written first. A failing test means the seam is not satisfied; the con
|
||||
## See Also
|
||||
|
||||
- [[Deployment Flow]]
|
||||
- [[index]]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: howto
|
||||
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
|
||||
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
|
||||
tags:
|
||||
- dankclaw
|
||||
- comfyui
|
||||
@@ -56,8 +56,8 @@ sampling), `sampler_name: euler`, `scheduler: simple`, `denoise: 1.0`, seed
|
||||
fixed in the file (`1930` in the art workflow, `2026` in the text one) —
|
||||
read from `dankclaw/comfyui/workflows/*.json` node `9`.
|
||||
|
||||
Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`,
|
||||
`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`,
|
||||
Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`
|
||||
`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`
|
||||
`SaveImage` (ids 3–11).
|
||||
|
||||
| Workflow | File | Shape | Steps | Made for |
|
||||
@@ -113,9 +113,9 @@ only `models`, `input`, `output`, `custom_nodes` and the healthcheck file
|
||||
Quantized preferred, sized for the 8060S iGPU (GTT-backed — `/system_stats`
|
||||
reports 122.4 GB of device VRAM on `cuda:0`, `curl http://127.0.0.1:8188/system_stats`
|
||||
2026-10-18; the RTX 3090 is frank-zealot's). All repos verified 2026-10-05 and
|
||||
all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`,
|
||||
`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`,
|
||||
`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`,
|
||||
all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`
|
||||
`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`
|
||||
`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`
|
||||
`huggingface.co/QuantStack/Wan2.2-TI2V-5B-GGUF`, 2026-10-18). None are on disk
|
||||
yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18).
|
||||
|
||||
@@ -126,7 +126,7 @@ yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18)
|
||||
| 3 | **DiffSynth ControlNets** (structure control) | `Comfy-Org/Qwen-Image-DiffSynth-ControlNets` | `split_files/model_patches/` canny + depth | ~2–3 GB each | Pose/edge/depth-guided layouts — pairs with the legible-text workflow for exact poster composition. The `QwenImageDiffsynthControlnet` node is already in the running ComfyUI (`GET http://127.0.0.1:8188/object_info/QwenImageDiffsynthControlnet` → 200, 2026-10-18) |
|
||||
| 4 | **Wan 2.2 TI2V-5B GGUF** (first video model) | `QuantStack/Wan2.2-TI2V-5B-GGUF` | Q4/Q5 GGUF (~6–8 GB) | ~6–8 GB | T2V+I2V in one dense 5B model; the gentlest on-ramp into video generation, runs in ComfyUI core (`wan` is in this build's `CLIPLoader` type enum — `GET http://127.0.0.1:8188/object_info/CLIPLoader` 2026-10-18) |
|
||||
|
||||
Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`,
|
||||
Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`
|
||||
#3 → `models/controlnet/` (create it), #4 → `models/diffusion_models/`.
|
||||
Fetch with `scripts/download-frank-*.sh` discipline (`-f` + byte-count +
|
||||
sha256 checks) or a plain `hf download` with the token in the environment —
|
||||
|
||||
@@ -43,7 +43,7 @@ Dankclaw is an AMD Strix Halo APU (gfx1151). Standard PyTorch ROCm wheels do not
|
||||
|
||||
## GPU device access (rootless), and why the unit exists
|
||||
|
||||
The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`,
|
||||
The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`
|
||||
`dankclaw/comfyui/quadlet/comfyui.container:45-46`) and uses
|
||||
`GroupAdd=keep-groups`. A literal `render` entry cannot work: rootless Podman
|
||||
cannot map the group name into the user namespace. The companion udev rule
|
||||
@@ -115,7 +115,7 @@ generated units.
|
||||
The web UI responds on `/` (200 `text/html`, `curl http://127.0.0.1:8188/`
|
||||
2026-10-18). ComfyUI also exposes `/system_stats`. On this APU, `/system_stats`
|
||||
reports `device=cuda:0 Radeon 8060S Graphics` (type `cuda`, the ROCm device). The
|
||||
health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`,
|
||||
health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`
|
||||
bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
|
||||
(`dankclaw/comfyui/quadlet/comfyui.container:52,57`); it GETs
|
||||
`http://127.0.0.1:8188/` inside the container, and the container reports
|
||||
@@ -135,5 +135,5 @@ bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
|
||||
- [[Dankclaw Stack]]
|
||||
- [[ComfyUI ROCm and Vulkan]] — the live-verified GPU path research
|
||||
- [[ComfyUI Qwen-Image Workflows]] — the Qwen-Image 2.1 payload, proven workflows, and recommended model/LoRA additions
|
||||
- [[Lemonade and sd.cpp]] — the ggml-family alternative
|
||||
- the ggml-family alternative
|
||||
- Knowledge base: [[ComfyUI]] (tools)
|
||||
|
||||
@@ -52,11 +52,10 @@ graph LR
|
||||
|
||||
The LLM rows are bare-metal systemd services, not containers, and this repo
|
||||
owns their scripts, units and knowledge base (see
|
||||
[LLM fleet knowledge base](../LLM/index.md)). Which payload and runtime sits on
|
||||
). Which payload and runtime sits on
|
||||
each port is era state and lives on the endpoint page, not here. `frank-npu`
|
||||
(the XDNA2 NPU endpoint) and `frank-medium` (the predecessor of `frank-archon`
|
||||
on the Dankserver port) are both retired — see [[frank-npu]] and
|
||||
[[frank-medium-radiance-mxfp4]].
|
||||
on the Dankserver port) are both retired — see and
|
||||
|
||||
## Boot Persistence
|
||||
|
||||
@@ -72,7 +71,7 @@ Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw`, open-
|
||||
|
||||
## See Also
|
||||
|
||||
- [[index]] — repository index
|
||||
- repository index
|
||||
- [[Network Architecture]] — routing and proxy
|
||||
- [[Observability]] — Grafana, Prometheus, logs
|
||||
- [LLM fleet knowledge base](../LLM/index.md) — the model-serving endpoints on this host
|
||||
- the model-serving endpoints on this host
|
||||
|
||||
@@ -1,35 +1,36 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
|
||||
description: "The engine behind the fleet's speech endpoint: its Vulkan build on the iGPU, its private bridge, and what it publishes."
|
||||
tags:
|
||||
- dankclaw
|
||||
- tts
|
||||
- voice
|
||||
- quadlet
|
||||
- cpu
|
||||
timestamp: 2026-10-18
|
||||
- vulkan
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Frank TTS Engine
|
||||
|
||||
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
|
||||
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** loaded once and synthesizing from there — **on the gfx1151 iGPU** (ggml's Vulkan backend, `--device Vulkan0`) since 2026-10-19, and in host RAM before that. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 when asked, and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
|
||||
|
||||
**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
|
||||
**Deployed 2026-10-18, cut over to the iGPU on 2026-10-19.** The first chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy shortly after the build, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
|
||||
|
||||
**Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
|
||||
|
||||
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host)
|
||||
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub and the CPU trial; the device rows re-verified on the live iGPU shape 2026-10-19)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Engine | `tts-server` from qwentts.cpp, pinned at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (`dankclaw/voiceservice/Containerfile.engine`). The source is **not vendored**: the image clones that commit and its recorded `ggml` submodule, so the build needs outbound HTTPS and the `cfhash` label is what keeps an unchanged re-deploy offline |
|
||||
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8` — the exact line the 2026-10-18 trial ran on this box. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both of which pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
|
||||
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree |
|
||||
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8`. **`GGML_VULKAN=ON` since 2026-10-19**, and that door needs three packages in the build stage: `libvulkan-dev`, `spirv-headers`, and **`glslc`** — which Ubuntu's `glslang-tools` does NOT ship (it ships `glslangValidator`, and `find_package(Vulkan COMPONENTS glslc REQUIRED)` fails without the package). The archive's `glslc` is 2023.8: `GL_KHR_cooperative_matrix` compiles, the coopmat2 / integer-dot / bf16 / micro-float shader families do not. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
|
||||
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), `libvulkan1` + `mesa-vulkan-drivers` (the loader and the RADV ICD — a loader with no ICD enumerates ZERO devices, and the server then dies in device lookup after both GGUFs have been read), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree. **No NVIDIA ICD on purpose**: exactly one Vulkan device in the enumeration is what makes the name `Vulkan0` safe to write down |
|
||||
| Talker | `qwen-talker-1.7b-base-Q4_K_M.gguf`, **1 219 245 248 B**, sha256 `ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d` (the Hub's LFS pointer oid) |
|
||||
| Tokenizer | `qwen-tokenizer-12hz-Q8_0.gguf`, **291 150 624 B**, sha256 `1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94` |
|
||||
| Source of both | `https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF`, fetched by `scripts/download-frank-tts-engine-qwentts.sh` into `/home/dank/models/Qwen3-TTS-GGUF`, mounted read-only at `/models`. It has to be the **base** variant: the engine refuses every voice registration unless the model reports type `base`, so on `customvoice`/`voicedesign` the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is **not** the decided artifact |
|
||||
| Memory | **5.7 GiB peak RSS** with both files resident (2026-10-18 trial). No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
|
||||
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish |
|
||||
| Device | `--device Vulkan0`: the gfx1151 iGPU, **shared with [[frank-dragoon]] as a co-tenant, never a handover**. Proven on the live pair, not inferred: `fuser -v /dev/dri/renderD129` lists `tts-server` beside `flash_serve`. Both gates (the RADV heap and dragoon's contention) are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`, which also carries the change surface. **Rollback: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, redeploy this one service** — the entrypoint then passes no `--device` at all, which is the CPU shape, no rebuild, nothing else restarts |
|
||||
| Memory | **1.7 GiB host RSS** since the iGPU cutover — the weights ride the device. No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
|
||||
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on the iGPU measures **0.20** (11.04 s of audio in 2.23 s) with `frank-dragoon` live on the same card, so a live stream has room to spare; a second concurrent synthesis would share the device with the first and make the caller behind it wait out the whole first utterance, so the FIFO stays a decision, not an accident |
|
||||
| Network | `systemd-frank-tts-engine` (10.89.5.0/24, `dankclaw/voiceservice/quadlet/frank-tts-engine.network`) and **nothing else**, with **no `PublishPort` at all**. `frank-tts` joins the same bridge and dials `http://frank-tts-engine:8880` |
|
||||
| Voices | `/home/dank/voices` mounted read-only at `/voices`, and it is the **warmer** that reads it, not the server: `tts-server` receives reference audio as base64 in a registration body and keeps its registry in memory only |
|
||||
| Health | `GET /health` → `{"status":"ok"}`, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (`HealthCmd`, 30 s interval, 120 s start period) **inside the container's namespace**, because there is no published port to probe from outside |
|
||||
@@ -37,18 +38,11 @@ timestamp: 2026-10-18
|
||||
|
||||
## Why this engine, and not the alternatives
|
||||
|
||||
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in [[Local TTS and Voice Cloning]], which is the source of these numbers):
|
||||
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. On that criterion the resident `tts-server` (arm A) beat the mainline per-request CLI path (arm B) on this box the same day, and the deciding property was **spread**: the resident server repeats the same first-audio latency every run, while the CLI path varies enough that some runs fall behind real time and a live listener hears the stream stall mid-sentence. That A/B was a CPU measurement and the endpoint has since moved to the iGPU, so the shape of the argument is what is kept here; no page carries its numbers any more, and they are not worth re-deriving.
|
||||
|
||||
| Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) |
|
||||
|---|---|---|
|
||||
| first audio byte | **86–451 ms** end to end; **94–112 ms** server-side | model load, per request |
|
||||
| warm real-time factor while streaming | **0.55–0.67** | 0.93–0.99 |
|
||||
| run-to-run spread | **±0.01** | ±0.25 |
|
||||
| HTTP surface | serves `/v1/audio/speech` itself | none — façade CLI only |
|
||||
The rollback to the other backend stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
|
||||
|
||||
The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
|
||||
|
||||
## Three laws this service lives under
|
||||
## Four laws this service lives under
|
||||
|
||||
**1. The registry law → the unit warms at boot.** The voice registry is **memory only**: a restart forgets every voice, and re-registering one does **not** warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why `frank-tts-engine.container` carries
|
||||
|
||||
@@ -64,10 +58,12 @@ frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms
|
||||
frank-tts-engine-warm: 3 voice(s) registered and warm
|
||||
```
|
||||
|
||||
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
|
||||
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's pool sizes and its timings, with no log line. The rule did not retire with the iGPU cutover: the binary still sizes those pools from what the kernel says, whatever the device ends up doing with the matmuls. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
|
||||
|
||||
**3. Streaming is not buffered.** At the same seed, `response_format=pcm` (chunked, codec chunks pushed as they decode) and `response_format=wav` (buffered) differ in **92.2 % of samples**, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across `response_format`. `frank-tts` always asks for `wav`; the warmer asks for `wav`; the test seam is forbidden from pinning either.
|
||||
|
||||
**4. The device law → the card is shared, and the rollback is one line.** Synthesis computes on the gfx1151 **beside** [[frank-dragoon]], so a change to either tenant is a change to both. Two gates decide that pair, and both passed on 2026-10-19 with the other tenant live: the RADV heap a rootless container tenant is actually handed (not the ~2 GiB DRM carve-out), and dragoon's own first-token and decode rates with a synthesis in flight — unchanged. Their numbers, the recipe that measures them, and the full change surface are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`; do not re-derive them here. What belongs on this page is the move: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, re-deploy this one service, and synthesis is back on CPU — no rebuild, and dragoon never restarts.
|
||||
|
||||
## The voice registry contract
|
||||
|
||||
A voice becomes known to the engine through `POST /v1/audio/voices`, and only these shapes are accepted:
|
||||
@@ -97,9 +93,9 @@ The **façade** on 7772 is the pair's public surface and carries the probe. Note
|
||||
| `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
|
||||
| `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
|
||||
| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, render access (`AddDevice=/dev/dri` + `GroupAdd=keep-groups`), health probe, warm hook, `Restart=always` |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
|
||||
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
|
||||
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law, the registry law and the device law in its header |
|
||||
| `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
|
||||
| `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
|
||||
| `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
|
||||
@@ -118,6 +114,25 @@ still open.
|
||||
6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
|
||||
7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
|
||||
|
||||
### 2026-10-19 — the iGPU cutover (PR #34, `6248f02`)
|
||||
|
||||
1. **Configure gate before touching anything** — the flag set, configured inside the
|
||||
digest-pinned base on Dankclaw: `Found Vulkan: … found components: glslc` then
|
||||
`Including Vulkan backend`. That probe is what caught `glslc` missing from
|
||||
`glslang-tools`; the Containerfile's flag comment carries the finding.
|
||||
2. **Deploy** — `./scripts/deploy.sh dankclaw --service frank-tts-engine`: image rebuild
|
||||
(the `cfhash` moved), unit reinstall, health gate, warm pass
|
||||
(`3 voice(s) registered and warm`), Playwright e2e 33/33.
|
||||
3. **Prove the device, not the flag** — `sudo fuser -v /dev/dri/renderD129` lists
|
||||
`tts-server` next to dragoon's `flash_serve`. Do this instead of reading the log: the
|
||||
engine prints nothing about which backend it chose, so argv (`--device Vulkan0`) proves
|
||||
intent and the render-node holder list proves use.
|
||||
4. **Consumer path** — one `/v1/audio/speech` per voice through the façade on 7772; all
|
||||
three answered with audio. The RTF figure in the Facts table comes from one engine-direct
|
||||
WAV, where the duration is exact.
|
||||
5. **Co-tenancy gate** — §5 of the verdict report: dragoon's first token and decode
|
||||
unchanged with a 16.9 s synthesis occupying the card.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Meaning | Move |
|
||||
@@ -128,6 +143,8 @@ still open.
|
||||
| Everything is ~2× slower after a systemd tweak | a cpuset/cap pin changed `hardware_concurrency()` and the engine resized its thread pools | remove the pin (law 2) |
|
||||
| `502` from the façade with `frank-tts upstream request failed` | the façade cannot resolve or reach the engine | `podman network inspect systemd-frank-tts-engine` and `podman inspect frank-tts --format '{{json .NetworkSettings.Networks}}'` — the façade must be attached to the bridge; a `127.0.0.1:8880` or `host.containers.internal:8880` upstream in `scripts/frank-tts.env` is the old shape and cannot work |
|
||||
| A voice is missing from the list | its directory has no `reference.wav`, or the name is not lowercase | fix the pack; both are refused loudly by design |
|
||||
| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see [[deployment]] |
|
||||
| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see |
|
||||
| The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this |
|
||||
| A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces |
|
||||
| Unit restarts every 10 s and the log ends on a **device** error *after* both GGUFs were read | the image has the Vulkan loader but no ICD, or the container has no render node — the device name then resolves to nothing | `podman exec frank-tts-engine ls /dev/dri` (it must see `renderD129`) and `podman exec frank-tts-engine dpkg -l \| grep -E 'mesa-vulkan\|libvulkan1'`; re-deploy to rebuild the image, or set `QWENTTS_DEVICE=` and re-deploy for the CPU shape |
|
||||
| `frank-dragoon` slows while TTS speaks | the contention premise broke — a Mesa bump, a third GPU tenant, a bigger synthesis | `QWENTTS_DEVICE=` + re-deploy this service, then re-run the gate recipe in §5 of `report/tts-npu-spike-verdict-2026-10-19.md` before putting it back |
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
|
||||
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
|
||||
tags:
|
||||
- endpoint
|
||||
- dankclaw
|
||||
- tts
|
||||
- voice
|
||||
- quadlet
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Frank TTS
|
||||
@@ -16,10 +18,10 @@ audio in that cloned voice. Everything else in the voice stack sits behind it
|
||||
the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
|
||||
never learn a path.
|
||||
|
||||
**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
|
||||
**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`
|
||||
both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
|
||||
the same change. Probes on the deployed endpoint: `/health` answers
|
||||
`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
|
||||
`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`
|
||||
and two different voices rendered the same sentence into **different** bytes
|
||||
(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
|
||||
pass-through of one default. The operator harness routes its `speech` role here;
|
||||
@@ -30,7 +32,7 @@ render needs the app installed, not this endpoint.
|
||||
| --- | --- |
|
||||
| Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
|
||||
| Port | **7772**, published (`PublishPort=7772:7772`) — the rest of the `frank-*` port family is in [[Dankclaw Stack]] |
|
||||
| Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
|
||||
| Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only. The request's `model` field is **ignored** (measured 2026-10-19: `voxcpm2` and `tts-1` both synthesize the same way) — the voice comes solely from `voice`, so a consumer that must name a model may name it anything |
|
||||
| Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
|
||||
| Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
|
||||
| Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
|
||||
@@ -109,6 +111,24 @@ own `/health` answers 200 with the backend name and the voice list it resolved
|
||||
without dialing the engine, so "façade healthy, engine down" is a real state: it
|
||||
shows up as a failed synthesis, never as a down probe.
|
||||
|
||||
## Where the engine computes
|
||||
|
||||
The **gfx1151 iGPU**, as a co-tenant beside `frank-dragoon`: the Vulkan build of
|
||||
`qwentts.cpp` with `--device Vulkan0`, and `/dev/dri` handed to the engine container. That is
|
||||
the shape since 2026-10-19. The flag, the device name, the render-node trap on this hybrid box
|
||||
and the one-line CPU rollback are owned by [[Frank TTS Engine]] — read that page before
|
||||
touching either tenant, because a change to one is a change to both. Co-tenancy was gated with
|
||||
`frank-dragoon` live on the card (its first token and decode rates unchanged with a synthesis
|
||||
in flight); both gate recipes are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`.
|
||||
|
||||
**The NPU is a door only for a model the vendor has already compiled for it.** XDNA2 runs nothing but
|
||||
precompiled AIE partition programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the
|
||||
ioctls, then declines this model family's graph without emitting a single log line. That is why TTS stays on
|
||||
the GPU — and it is not a general veto on the device: halogen ships its own precompiled catalog (embeddings,
|
||||
decisions, rerankers) that does run there, which is what [[frank-dragoon]]'s NPU-lane section is about. The
|
||||
measured record and the model-side contract live in `report/` at the repo root
|
||||
(`tts-npu-spike-verdict-2026-10-19.md`).
|
||||
|
||||
## Files
|
||||
|
||||
| Path | What |
|
||||
@@ -145,7 +165,7 @@ A `/health` probe job would therefore be structurally never-green, and because
|
||||
`LlmEndpointDown` is scoped across the whole `frank-*` job family (the rule set
|
||||
lives on [[Prometheus]]), a permanently red `frank-tts` target would page on
|
||||
Signal every few minutes for an endpoint that is fine. The engine cannot be
|
||||
scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
|
||||
scraped at all: it publishes nothing. Status lives in `systemctl --user status`
|
||||
`podman healthcheck run`, the two unit journals, and the homepage card, which
|
||||
probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
|
||||
2026-10-18 deploy.
|
||||
@@ -153,7 +173,7 @@ probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
|
||||
## The harness speech role
|
||||
|
||||
The operator harness reaches this endpoint through
|
||||
`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
|
||||
`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`
|
||||
`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
|
||||
`openai-speech` on the **model** row (naming a runner api at *provider* level
|
||||
fails the whole file's validation and disables every custom provider, which is
|
||||
@@ -169,7 +189,7 @@ harness bundle and measured the day of the flip:
|
||||
`speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
|
||||
and it never reaches an HTTP request. So a harness call arrives voice-less by
|
||||
default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
|
||||
pack answers a voice-less request and the journal marks that line `(default)`,
|
||||
pack answers a voice-less request and the journal marks that line `(default)`
|
||||
while an **unknown** name still 400s with the list that exists.
|
||||
- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
|
||||
select a pack; it hands the vocalizer a value outside its enum. (The first
|
||||
@@ -202,3 +222,4 @@ harness bundle and measured the day of the flip:
|
||||
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
||||
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
||||
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
|
||||
---
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Gitea
|
||||
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
|
||||
tags:
|
||||
@@ -33,7 +33,7 @@ The containers share the private bridge network `systemd-gitea`. The quadlet net
|
||||
| `dank/command-center` | second public repo on the instance (`GET http://127.0.0.1:3000/api/v1/repos/search` 2026-10-18) |
|
||||
| `dank/vllmScripts` | retired 2026-08-22 — merged into `dank/serverAppConfigs` (history preserved) *(unverified 2026-10-18: the anonymous API search returns no match and `/dank/vllmScripts` is 404, so whether the repo still exists privately cannot be checked without a token)* |
|
||||
|
||||
The LLM fleet knowledge base: [serverAppConfigs-docs/LLM/index.md](../LLM/index.md)
|
||||
The LLM fleet knowledge base:
|
||||
|
||||
## Management
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ entrypoint does not start the dashboard. The dashboard is a separate process, so
|
||||
it runs in its own Quadlet unit
|
||||
([`dankclaw/hermes/quadlet/hermes-dashboard.container`](../../dankclaw/hermes/quadlet/hermes-dashboard.container):
|
||||
image `localhost/hermes-agent:latest`, `Pull=never`, `Exec=hermes dashboard
|
||||
--host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`,
|
||||
--host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`
|
||||
`RestartSec=5`). All three Hermes containers mount the same `/data` bind mount.
|
||||
|
||||
Unlike its two siblings — `hermes.container` and `hermes-webui.container` both
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Hermes-WebUI (Hermex server)
|
||||
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
|
||||
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
|
||||
tags:
|
||||
- hermes
|
||||
- hermes-webui
|
||||
@@ -40,7 +40,7 @@ from another LAN host: `/health` → 200, `/` → 302) — **Vhost:**
|
||||
|
||||
## Why a container, not a host process
|
||||
|
||||
The gateway's state files are owned by the container-mapped uid (`109999`,
|
||||
The gateway's state files are owned by the container-mapped uid (`109999`
|
||||
from rootless Podman's `1 → 100000/65536` subuid range — `/etc/subuid` →
|
||||
`dank:100000:65536`, and the agent's runtime uid is 10000, so 100000 + 9999 =
|
||||
109999; `ls -ld /home/dank/Containers/hermes/data` → uid 109999, 2026-10-18). A fresh rootless
|
||||
@@ -82,12 +82,12 @@ the sibling dashboard on `9119` is not.
|
||||
|
||||
- Dedicated `tunnel` account on Dankclaw (`getent passwd tunnel` → uid 994,
|
||||
shell `/usr/sbin/nologin`, 2026-10-18; password locked). Authentication is key-only
|
||||
*(unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
|
||||
- (unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
|
||||
without `sudo`)*; the account can do nothing except
|
||||
authenticate — `ForceCommand /usr/sbin/nologin` kills any session channel
|
||||
while the transport stays up for port forwarding.
|
||||
- sshd drop-in `/etc/ssh/sshd_config.d/99-tunnel.conf` (exists, 118 B, 2026-08-31)
|
||||
scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`,
|
||||
scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`
|
||||
`X11Forwarding no` and `ForceCommand /usr/sbin/nologin` — all four lines are
|
||||
present under `Match User tunnel` (`cat` 2026-10-18).
|
||||
- Key options on the authorized key pin the blast radius:
|
||||
@@ -130,7 +130,7 @@ only if inactive (it is never restarted on deploy, so in-flight browser
|
||||
sessions survive), re-tags the image alias, then restarts
|
||||
`hermes.service`, `hermes-dashboard.service` and
|
||||
`hermes-webui.service` in the **user** manager of `dank` (linger must be on for
|
||||
them to come up at boot — `loginctl show-user dank -p Linger` → `yes`,
|
||||
them to come up at boot — `loginctl show-user dank -p Linger` → `yes`
|
||||
2026-10-18), so gateway/dashboard/webui restarts stay isolated from
|
||||
inference deploys.
|
||||
Before first deploy, set a strong `HERMES_WEBUI_PASSWORD` in the live
|
||||
|
||||
@@ -18,7 +18,7 @@ bindings, 2026-10-18). Note the **container name drift**: the unit is
|
||||
`podman run --name systemd-%N …` (`systemctl --user cat hermes.service`), so
|
||||
`podman ps` shows the gateway as **`systemd-hermes`** — the repo unit sets no
|
||||
`ContainerName=` key even though its header comment says "container name:
|
||||
hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached via
|
||||
hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached
|
||||
`hermes.franksdank.cloud` — internal-only by DNS absence (no public Porkbun A
|
||||
record; the name resolves from the Pi-hole hosts file,
|
||||
`dankclaw/pihole/addn-hosts:39`) — **Docs:**
|
||||
@@ -37,7 +37,7 @@ designed wiring; the remedy for both halves is in the log line
|
||||
|
||||
**Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted
|
||||
at `/data` in all three Hermes containers (gateway, webui, dashboard —
|
||||
`hermes.container:36`, `hermes-webui.container:46`,
|
||||
`hermes.container:36`, `hermes-webui.container:46`
|
||||
`hermes-dashboard.container:24`). Each
|
||||
container additionally gets an anonymous volume at `/opt/data`, which is
|
||||
re-created empty on every container recreation — anything that lives only there
|
||||
@@ -80,7 +80,7 @@ is not durable. The tracked, sanitized configuration baseline is
|
||||
`frank-dragoon` — verified from the host with `curl
|
||||
http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
|
||||
`127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
|
||||
and is IPv4-only);
|
||||
and is IPv4-only)
|
||||
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
|
||||
2026-10-18, that endpoint serves only `frank-archon`** (`GET
|
||||
http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
|
||||
@@ -115,7 +115,7 @@ per session; the browser itself lives in a dedicated daemon container, the
|
||||
digest-pinned headless-shell unit `chrome-devtools-protocol.container` (host
|
||||
network, loopback-only bind on 127.0.0.1:9222 — `ss -ltnp` 2026-10-18 shows
|
||||
`127.0.0.1:9222 headless-shell` and nothing on `0.0.0.0`; image
|
||||
`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`,
|
||||
`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`
|
||||
`dankclaw/hermes/quadlet/chrome-devtools-protocol.container:28`). Both consumers — the Signal
|
||||
gateway and the webui server — run in the host network namespace (`podman
|
||||
inspect` → `net=host` for both, 2026-10-18), so they can
|
||||
@@ -248,7 +248,7 @@ What actually breaks, and the fix:
|
||||
effort: none # mandatory, see below
|
||||
```
|
||||
|
||||
Before copying this: which id is served on which port is endpoint state — see
|
||||
Before copying this: which id is served on which port is endpoint state —
|
||||
[[frank-zealot]] and [[frank-archon]]. The provider/model
|
||||
pairing above is the shape that was landed in 2026; re-derive it from the live
|
||||
config before reusing it.
|
||||
@@ -282,7 +282,7 @@ Auth contract (verified live 2026-08-31, re-checked 2026-10-18): `POST /api/auth
|
||||
on match, 401 on miss. No username field. `/health` answers 200
|
||||
unauthenticated (`curl http://127.0.0.1:8787/health` → 200 with
|
||||
`{"status":"ok", …}` today) and `/` answers 302 (`curl -o /dev/null -w
|
||||
%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth via
|
||||
%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth
|
||||
those**; probe
|
||||
login with a deliberately wrong password (expect 401) and the real one
|
||||
(expect 200). Details on the webui surface: [[Hermes-WebUI]].
|
||||
@@ -311,7 +311,7 @@ login with a deliberately wrong password (expect 401) and the real one
|
||||
instead of loopback" — live bind `0.0.0.0:8787`, `ss -ltnp` 2026-10-18;
|
||||
off-network ingress is SSH tunnel only —
|
||||
user `tunnel`, port **22** (`sshd -T` → `port 22`, 2026-10-18), key-only auth
|
||||
*(unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*,
|
||||
- (unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*
|
||||
forward remote end
|
||||
localhost:8787).
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Marreta on Dankclaw
|
||||
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
|
||||
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
|
||||
tags:
|
||||
- dankclaw
|
||||
- marreta
|
||||
@@ -19,7 +19,7 @@ returns a stripped, readable version of the page — no paywall chrome, no
|
||||
tracking, no cookie walls. It is a Laravel 13 app on the
|
||||
`serversideup/php:8.4-fpm-nginx` image (`composer.json` requires
|
||||
`php: ^8.4`, `laravel/framework: ^13.0`, `filament/filament: ^5.7` —
|
||||
<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>),
|
||||
<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>)
|
||||
with an optional headless-browser companion for JavaScript-heavy pages.
|
||||
|
||||
**Host:** Dankclaw — **vhost:** `marreta.franksdank.cloud` (Caddy, guard-chained)
|
||||
@@ -31,7 +31,7 @@ with an optional headless-browser companion for JavaScript-heavy pages.
|
||||
- Fetches a target URL with a cascade of strategies: plain HTTP (curl), a
|
||||
headless browser (Lightpanda) for JS pages, and the Internet Archive
|
||||
(Wayback) as a last resort — the three `app/Services/Fetch/*FetchStrategy.php`
|
||||
files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`,
|
||||
files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`
|
||||
`WaybackFetchStrategy`, dispatched by `app/Services/FetchManager.php`).
|
||||
- Strips paywall/consent/tracking markup and re-renders clean HTML.
|
||||
- Ships a Filament admin panel at `/admin` (login = `ADMIN_EMAIL` /
|
||||
@@ -69,7 +69,7 @@ graph LR
|
||||
| `marreta-browser.service` | `dankclaw/marreta/quadlet/marreta-browser.container` | `docker.io/lightpanda/browser:nightly` (`:26`) | `gluetun.container` (`:25`) | `9222` (CDP) |
|
||||
|
||||
Both units are boot-persistent (`WantedBy=default.target`) and self-healing
|
||||
(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`,
|
||||
(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`
|
||||
`marreta-browser.container:30-34`), per the repo's Boot Persistence rule. Live
|
||||
2026-10-18 both report `systemctl --user is-enabled` → `generated` and
|
||||
`is-active` → `active`, and `podman inspect` reports marreta **healthy**
|
||||
@@ -87,14 +87,14 @@ The fix is a single env var, verified against the serversideup source:
|
||||
|
||||
- The image sets `ENV NGINX_HTTP_PORT="8080"` and the site template is
|
||||
`listen ${NGINX_HTTP_PORT} default_server;`.
|
||||
- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build);
|
||||
- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build)
|
||||
the entrypoint `10-init-webserver-config.sh` runs `envsubst` over every
|
||||
`NGINX_*` variable **at container start**.
|
||||
- So `Environment=NGINX_HTTP_PORT=8085` in the quadlet relocates the listen
|
||||
port off the collision with no custom build.
|
||||
|
||||
Marreta then publishes **nothing** of its own — the netns owner (Gluetun)
|
||||
publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`),
|
||||
publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`)
|
||||
exactly as it publishes `8081:8080` for SearXNG. Caddy dials
|
||||
`192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`). Live env confirms the
|
||||
relocation: `podman inspect marreta` carries `NGINX_HTTP_PORT=8085`
|
||||
@@ -127,7 +127,7 @@ on the mechanism: `BrowserFetchStrategy.php` only calls
|
||||
(`dankclaw/Caddyfile:159-163`).
|
||||
2. **Auth decision** — Anubis dials Caddy's `http://:8016`, which proxies
|
||||
whole-request to `authentik-server:9000` (`dankclaw/Caddyfile:22-24,94`).
|
||||
3. **Post-auth routing** — authenticated traffic returns to `http://:8017`,
|
||||
3. **Post-auth routing** — authenticated traffic returns to `http://:8017`
|
||||
where `@marreta host marreta.franksdank.cloud` routes to
|
||||
`reverse_proxy 192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`).
|
||||
|
||||
@@ -164,7 +164,7 @@ Marreta ships `trustProxies(at: '172.16.0.0/12,10.0.0.0/8,192.168.0.0/16,127.0.0
|
||||
— that is the `TRUSTED_PROXIES` fallback in upstream `bootstrap/app.php`, and
|
||||
this container sets no `TRUSTED_PROXIES`, so the default is what runs
|
||||
(`podman inspect marreta` 2026-10-18) — and forces `https` when `APP_URL`
|
||||
starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`,
|
||||
starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`
|
||||
`URL::forceScheme('https')`). With
|
||||
`APP_URL=https://marreta.franksdank.cloud` (`dankclaw/marreta/quadlet/marreta.container:41`) the app honors the forwarded
|
||||
scheme and client IP through the whole chain, so there is no redirect loop
|
||||
@@ -234,7 +234,7 @@ netns down and would otherwise orphan it).
|
||||
Deployed via `deploy.sh dankclaw --service gluetun --service marreta --service
|
||||
caddy` (gluetun first to recreate the tunnel with the new `8082:8085` publish,
|
||||
which also cycles SearXNG's netns ref). Authentik `ProxyProvider` `marreta`
|
||||
registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
|
||||
registered (pk 13, `external_host=https://marreta.franksdank.cloud/`
|
||||
`internal_host=http://caddy:8017`). Results:
|
||||
|
||||
- Unauth edge → Anubis challenge page (not 502). TLS cert issued via DNS-01.
|
||||
@@ -245,14 +245,14 @@ registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
|
||||
(`GET 127.0.0.1:9222/json/version` → `Lightpanda/1.0`).
|
||||
- SearXNG unaffected: `:8081` → 200 after the tunnel recreate.
|
||||
|
||||
> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`,
|
||||
> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`
|
||||
> which 500s every app route (see Secrets above). Fixed by re-keying with the
|
||||
> `base64:` prefix before any encrypted data was persisted.
|
||||
|
||||
## Citations
|
||||
|
||||
- Marreta upstream: <https://github.com/manualdousuario/marreta> (README,
|
||||
`compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`,
|
||||
`compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`
|
||||
`lang/` directory listing).
|
||||
- serversideup/php fpm-nginx port mechanism: `serversideup/docker-php`
|
||||
`src/variations/fpm-nginx/Dockerfile` (`NGINX_HTTP_PORT="8080"`) and
|
||||
|
||||
@@ -1,30 +1,31 @@
|
||||
---
|
||||
type: reference
|
||||
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
|
||||
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
|
||||
tags:
|
||||
- dankclaw
|
||||
- open-notebook
|
||||
- podcasts
|
||||
- quadlet
|
||||
timestamp: 2026-10-18
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Open Notebook
|
||||
|
||||
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
||||
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-archon**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
||||
|
||||
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
|
||||
**Deployed 2026-10-19.** Both units installed and `active`, both containers healthy, SurrealDB migrated to v25 on first boot, and the first episode rendered end to end: a note in, an outline, 23 dialogue turns, 23 clips, and one **747-second MP3 on the host bind, produced in 301 seconds** — about 2.5× real time with every concurrency knob at 1. The download route answers `200 audio/mpeg` with the bearer token and **401 without it**. It is guarded like every other user service — `https://notebook.franksdank.cloud`, Anubis → Authentik → the app's own login — and the API is no longer published on the LAN at all.
|
||||
|
||||
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
||||
**Host:** Dankclaw (rootless Podman, user `dank`) — **User path:** `https://notebook.franksdank.cloud` (guarded) — **Published:** `0.0.0.0:8502` (the only upstream Caddy dials) and `127.0.0.1:5056` (the deployer's health gate, nothing else) — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
||||
|
||||
## Facts (verified 2026-10-18 against the pinned images, upstream source, and the live host)
|
||||
## Facts (images and upstream source verified 2026-10-18; live-host columns re-read 2026-10-19 after deploy)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| App image | `docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212` (`open-notebook.container:79`) — the tag upstream's own `docker-compose.yml:27` pins for the multi-container shape. `v1-latest` is a moving tag, so the manifest-list digest (resolved from the registry 2026-10-18) is what makes this reproducible |
|
||||
| Database image | `docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723` (`open-notebook-surrealdb.container:66`) — that digest is **SurrealDB 2.7.0** (image built 2026-09-23; measured by running the digest: `surreal version`) |
|
||||
| Do NOT use | the `-single` tags (`v1-latest-single`). That variant bundles SurrealDB inside the app image and is deprecated upstream; here it would give the host a second, hidden database |
|
||||
| Ports | container 8502 (Next.js UI) + 5055 (FastAPI API) → published `0.0.0.0:8502:8502` and `0.0.0.0:5056:5055` (`open-notebook.container:83-84`). **Host 5055 is taken**: seerr publishes `127.0.0.1:5055` (`dankclaw/seerr/quadlet/seerr.container`, and `ss -ltn` on Dankclaw shows `127.0.0.1:5055` live, 2026-10-18). The API therefore leaves on 5056, which is what forces `API_URL` (below) |
|
||||
| Ports | container 8502 (Next.js UI) + 5055 (FastAPI API) → published `0.0.0.0:8502:8502` and `127.0.0.1:5056:5055` (`open-notebook.container`). **Host 5055 is taken**: seerr publishes `127.0.0.1:5055` (`dankclaw/seerr/quadlet/seerr.container`, and `ss -ltn` on Dankclaw shows it live). The API therefore leaves on 5056, and on loopback only: the vhost is its sole user path and the image's own Next.js layer rewrites `/api/*` to it, so nothing outside the container needs it |
|
||||
| User path | `https://notebook.franksdank.cloud` → `anubis:8923` → post-auth listener `@notebook` → `192.168.0.143:8502` (`dankclaw/Caddyfile`); Authentik provider `notebook-proxy` (`external_host` the vhost with its trailing slash, `internal_host http://caddy:8017`); LAN record in `dankclaw/pihole/addn-hosts` |
|
||||
| Database port | 8000 inside its own namespace, **never published** (`open-notebook-surrealdb.container` has no `PublishPort`). Upstream publishes `127.0.0.1:8000` for debugging (`docker-compose.yml:18`); this unit does not, and the seam asserts that it stays that way |
|
||||
| Data | `open-notebook-data:/app/data` (named volume: uploads, LangGraph SQLite checkpoints, extraction caches) **plus** the interim host bind `/home/dank/media/podcasts:/app/data/podcasts` (`open-notebook.container:87-88`); database in the `open-notebook-db` named volume |
|
||||
| Networks | app: `dankclaw_default` first, then the private `systemd-open-notebook` (10.89.4.0/24, `open-notebook.network:33`); database: the private bridge only. Order is load-bearing — the generated `--network` flags keep file order, so the default route rides the network where `host.containers.internal` is proven |
|
||||
@@ -33,6 +34,17 @@ open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook))
|
||||
| Env | `scripts/open-notebook.env` (repo, knobs) then `/home/dank/Containers/open-notebook/env` (host, secrets, mode 600) — two `EnvironmentFile=` lines per unit, repo file **first** because the last file wins |
|
||||
| Deployer | `deploy_open_notebook` in `scripts/deploy-remote.sh`: seed secrets → ensure both digest pins are in local storage → install both units + the network → `mkdir -p /home/dank/media/podcasts` → restart database, wait for its container health → restart app → `wait_for_health open-notebook http://127.0.0.1:5056/health 180` |
|
||||
|
||||
## The guard chain, and the single origin
|
||||
|
||||
`notebook.franksdank.cloud` follows the fleet's exposure policy in this order, and the order is the law ([[Network Architecture]]):
|
||||
|
||||
1. **Provider first.** Authentik proxy provider `notebook-proxy` existed before any route pointed at the app: `external_host https://notebook.franksdank.cloud/` (exact vhost, trailing slash), `internal_host http://caddy:8017`, the three flows copied from `embedded-outpost`, Application `Open Notebook` (`notebook-guard`), and the provider attached to the embedded outpost's provider set. Flip the vhost first and authenticated traffic lands on the outpost's error page. Verify after the change that the outpost logged no `Failed to fetch outpost from API` (the null-flow wedge) and that an already-guarded vhost still answers.
|
||||
2. **Then the route, in one deploy:** the guarded site block (`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}` — the whole request, so no path ever reaches the app unchallenged) plus the post-auth `@notebook` handler on `http://:8017`, plus the LAN record in `dankclaw/pihole/addn-hosts`. The public Porkbun A record is a host action, not a repo file.
|
||||
3. **One upstream, not two.** `reverse_proxy 192.168.0.143:8502`: the image's own Next.js layer rewrites `/api/*` to the FastAPI beside it (`next.config.ts`, `INTERNAL_API_URL` default `http://localhost:5055`), so the API needs no route and no outward face — hence `127.0.0.1:5056`. Caddy dials the published LAN port rather than a container name because this Caddy lives in root's Podman and cannot see a rootless container at all ([[Container Network Namespaces]]). SSE (`/api/search/ask`, chat) streams through the same bare proxy that carries OpenWebUI's chat at `ai.franksdank.cloud`; no `flush_interval` or timeout directive is needed, and adding a route timeout is precisely what would cut it.
|
||||
4. **`API_URL` must name the vhost.** `GET /config` (the Next route) answers in priority order: `API_URL` env → autodetect from `Host` + `X-Forwarded-Proto` → `http://localhost:5055`. Autodetect appends **`:5055`**, a port that belongs to seerr on this host, so leaving `API_URL` unset hands the browser an address nothing answers and every call fails under a UI that loaded fine. `CORS_ORIGINS` is set to the same single origin only to keep the app from defaulting to `*`. `scripts/open-notebook.env` carries both.
|
||||
|
||||
The app's own login stays on behind the chain. It is not redundant: publishing the API on loopback does **not** hide it from this host's other containers, which can dial `open-notebook:5055` by name on `dankclaw_default`. The chain decides *who from outside* gets in; `OPEN_NOTEBOOK_PASSWORD` decides what anything that already reached the port may do.
|
||||
|
||||
## What the image actually runs
|
||||
|
||||
One container, three processes under supervisord (`supervisord.conf` upstream): `uvicorn api.main:app --port 5055` (`:12`), the job worker (`:25`), and the Next.js frontend (`:39`). Two consequences the config depends on:
|
||||
@@ -44,25 +56,43 @@ Also from the image itself: `WORKDIR /app` and `EXPOSE 8502 5055` (`Dockerfile:8
|
||||
|
||||
`DATA_FOLDER` is hard-coded `"./data"` (`open_notebook/config.py:5`) and `PODCASTS_FOLDER` is `"./data/podcasts"` (`:19`); the database stores each episode's audio path **relative** to that folder (`episodes/<uuid>/audio/<uuid>.mp3`), and the download route is `GET /api/podcasts/episodes/{id}/audio`. That is why covering one subdirectory with a bind mount is sufficient — nothing in the database has to change if the mount moves.
|
||||
|
||||
## Provisioning the providers (after first boot; UI or REST, never a unit)
|
||||
## What is provisioned (REST, 2026-10-19)
|
||||
|
||||
Credentials are stored in the database, encrypted, and are **not expressible in env or a Quadlet file**. The sequence is one-time, in **Manage → Models**, and both entries use provider type **OpenAI Compatible** (upstream: any server that implements `/v1/audio/speech` works this way — `docs/5-CONFIGURATION/local-tts.md:5`).
|
||||
Credentials live in the database, encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`, and are **not expressible in env or a Quadlet file** — the UI (**Manage → Models**) or the REST API are the only doors. Both use provider type `openai_compatible`, because any server that answers `/v1/audio/speech` or `/v1/chat/completions` works through it (`docs/5-CONFIGURATION/local-tts.md:5`).
|
||||
|
||||
1. **TTS credential** — base URL **`http://frank-tts:7772/v1`**. The `/v1` is part of the URL: the client posts to `<base_url>/audio/speech`, so a base URL without `/v1` 404s on every dialogue line (`local-tts.md:72-80`, the base-URL table, always carries `/v1`). `frank-tts` resolves by container name because the app is attached to `dankclaw_default`, the network frank-tts's unit joins (`dankclaw/voiceservice/quadlet/frank-tts.container:51-52`). If frank-tts is ever moved off that network, the published-port form is `http://192.168.0.143:7772/v1` — a published port is dialable at the host's LAN address from any netns. The API-key field stays empty; frank-tts does not authenticate.
|
||||
2. **LLM credential** — base URL **`http://host.containers.internal:7771/v1`**, model row `frank-zealot`. frank-zealot is a **host process** on `0.0.0.0:7771` (`ss -ltn` on Dankclaw shows the listener with no container owning it, 2026-10-18), and on this netavark host a bridge container reaches the host's LAN IP only at a *published* port — which is why Prometheus scrapes the same endpoint as `host.containers.internal:7771` (`shared/prometheus/prometheus.yml:89-111`, "not their own LAN addresses"). See [[Container Network Namespaces]]. `http://192.168.0.143:7771/v1` is the variant to try only if the gateway name ever stops resolving; it is expected to be refused from a bridge netns.
|
||||
3. **Model rows are created by name.** Discovery is optional (see the next section): type the model id and confirm the add. One base URL per credential, so the TTS and LLM credentials are necessarily two separate configurations.
|
||||
| Record | Value |
|
||||
|--------|-------|
|
||||
| Credential `frank-tts` | modality `text_to_speech`, base URL `http://frank-tts:7772/v1`, no API key (the shim does not authenticate) |
|
||||
| Credential `frank-archon` | modality `language`, base URL `http://192.168.0.235:7770/v1` |
|
||||
| Model row `voxcpm2` | type `text_to_speech` on the frank-tts credential. The name is decorative: frank-tts ignores the request's `model` field and synthesizes whatever `voice` names ([[Frank TTS]]) |
|
||||
| Model row `frank-archon` | type `language`; `PUT /api/models/defaults` points chat, transformation and large-context at it. **The embedding default stays null — nothing in the fleet serves `/v1/embeddings`** |
|
||||
| Speaker profile `dank-duo` | two speakers, Frank → `voice_id: narrator1`, Comrade → `narrator2`, `voice_model` = the TTS row |
|
||||
| Episode profile `dank-daily` | `num_segments: 4`, `speaker_config` = `dank-duo`, outline and transcript LLM = the archon row, briefing that ends "Never invent numbers." |
|
||||
|
||||
**Verify reachability before saving**, from inside the container, because both addresses are netns-dependent:
|
||||
Order that works over REST: credentials → model rows (`POST /api/models` with `credential: <credential id>`) → `PUT /api/models/defaults` → speaker profile (`voice_model: <model id>`) → episode profile (`speaker_config` / `outline_llm` / `transcript_llm`). `POST /api/models/{id}/test` proves each row: the language row answered with a real completion from frank-archon; the TTS row reached frank-tts and came back `unknown voice 'alloy'; available voices: narrator1, narrator2, narrator3` — the app's own test path hard-codes an OpenAI voice, so that message *is* the proof the leg works.
|
||||
|
||||
**`POST /api/podcasts/generate` takes profile NAMES, not record ids.** `episode_profile: "dank-daily"` works; the `episode_profile:…` id returns a bare `500 Failed to submit podcast generation job`, and the actual reason (`Episode profile '<id>' not found`) appears only in `podman logs open-notebook`.
|
||||
|
||||
**The scriptwriter is frank-archon, not the planned frank-zealot, because two rules collide on the address.** Measured from inside the app's netns 2026-10-19:
|
||||
|
||||
- `192.168.0.143:7771` (frank-zealot, a host-netns listener) and `192.168.0.143:7770` (frank-dragoon, a **root**-published port) both refuse the connection; `host.containers.internal:7771` answers. Full census: [[Container Network Namespaces]].
|
||||
- The app refuses to **store** that one working address: its URL guard rejects any base URL that resolves inside link-local `169.254.0.0/16` (`open_notebook/utils/url_validation.py`, "Link-local addresses are not allowed (metadata endpoints)"), unconditionally — no env var touches it, and `AddHost=` cannot dodge it because the check runs on the resolved address.
|
||||
|
||||
`http://192.168.0.235:7770/v1` — another machine, plain RFC1918 — is the only fleet LLM address that passes both tests. Cost: frank-archon runs `--max-num-seqs 2` and the operator session itself lives there, so a render is a slow-down for whoever else is talking to that endpoint, not a stall; `ESPERANTO_LLM_TIMEOUT=180` covers an outline or transcript call comfortably.
|
||||
|
||||
**frank-dragoon can write transcripts, but only greedily — and open-notebook cannot ask for greedy.** Measured 2026-10-19 against the live endpoint: `response_format` as `json_schema` or `json_object` returns schema-valid JSON at `temperature: 0` and a **400** at any sampled temperature — halogen decodes greedy under a schema, and because its sampler default is server-side, omitting `temperature` is sampled, not greedy. That 400 is the trap: esperanto's `is_json_schema_unsupported_error` matches the words *not implemented* in it, so `_SchemaFallbackModel` in `podcast_creator/nodes.py` reads it as "this endpoint has no schema support", downgrades to `json_object`, gets the same 400, and **fails the run** — esperanto's own rescue (drop `response_format` entirely) fires only on the different message `'response_format.type' must be 'json_schema'`. The full law is on [[frank-dragoon]]. And the fix is not reachable from here: a model record is `{credential, name, provider, type}` and a model response adds only timestamps (`GET /openapi.json`, `ModelCreate`/`ModelResponse`) — **there is no per-row config field**, no way to attach `temperature: 0` to one row from the UI or the API. Making `frank-dragoon` the writer would therefore need something in the path that rewrites request bodies, which is a heavier tenant than the contention it removes. **frank-archon stays the scriptwriter.** Reachability would be a separate problem anyway: `frank-dragoon` is not on `dankclaw_default` (the serve script passes no `--network`), so nothing on this network has a name for it, and the one address that does answer — `host.containers.internal:7770` — is the link-local the guard refuses to store ([[Container Network Namespaces]]).
|
||||
|
||||
**No embedding model is deployed, and saving a note is where that shows up.** Every note save submits an `embed_note` command that fails in the worker with `No embedding model configured. Please configure one in the Models section.` The note itself is stored and readable; semantic search, source ingestion and notebook-context questions stay dead until something serves `/v1/embeddings`. Podcast generation from `content` does not need embeddings, which is why renders work today. The cheapest door is **halogen's NPU lane, not a new endpoint**: `qwen3-embedding-0.6b` is already in the catalog of the image `frank-dragoon` runs and answers `/v1/embeddings` on the same port (1024 unit-length floats; the instruct prefix goes in the text, and `check_embedding_ctx_length=False` for LangChain). What enabling it costs — the accel device, the host XRT mount, and the fabric-clock hold that GPU+NPU co-run requires — is on [[frank-dragoon]]; after that, the embedding row still has to clear the reachability rule above.
|
||||
|
||||
**Secrets never go in a command line.** The API's bearer token *is* `OPEN_NOTEBOOK_PASSWORD`, and a `-H "Authorization: Bearer …"` argument is world-readable in `/proc/<pid>/cmdline`; put it in a mode-600 curl config and pass `-K`:
|
||||
|
||||
```bash
|
||||
podman exec open-notebook curl -sS http://frank-tts:7772/health
|
||||
podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
|
||||
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
|
||||
curl -s -K "$cfg" http://127.0.0.1:5056/api/models | head -c 400
|
||||
rm -f "$cfg"
|
||||
```
|
||||
|
||||
**The blocker moved; it did not vanish.** frank-tts and frank-tts-engine are **deployed** (2026-10-18) and answer on 7772, so the voice leg works end to end — `podman exec open-notebook curl -sS http://frank-tts:7772/health` is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.
|
||||
|
||||
**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). The decision rests on each endpoint's documented request surface ([[frank-zealot]], [[frank-dragoon]]): zealot's is a known-good target for that call shape, dragoon's structured-output behaviour is **unverified**, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
|
||||
|
||||
## Speakers and voices: 1–4, and the IDs are free text
|
||||
|
||||
A **speaker profile** carries one to four speakers (`docs/3-USER-GUIDE/creating-podcasts.md:27`), each with a personality and a **Voice ID that is typed exactly and passed through to the TTS request untouched** (`:29`) — there is no whitelist, no enum, no validation against a voice list. So the fleet's voice-pack names work verbatim:
|
||||
@@ -92,9 +122,11 @@ Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-
|
||||
|
||||
The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
|
||||
|
||||
What the guard costs, measured on that first render (the deployed shape): 23 dialogue clips, one POST each, **10–11 s per clip** with both knobs at 1 → 301 s of wall clock for 747 s of speech, i.e. **2.5× real time**, box still responsive. That ratio is the budget to hold any future TTS engine to: halve the per-clip time, halve the episode.
|
||||
|
||||
## Secrets, and the difference between blank and unset
|
||||
|
||||
`OPEN_NOTEBOOK_PASSWORD` unset means **authentication off** (`environment-reference.md:46`) — every API route, including the episode-audio download, answers anyone who can reach the published port. That is a real exposure on a LAN-published service with no vhost and no guard chain in front of it, which is why `deploy_open_notebook` refuses to start either unit unless the seeded host file carries a value for all four secret keys (`OPEN_NOTEBOOK_ENCRYPTION_KEY`, `OPEN_NOTEBOOK_PASSWORD`, `SURREAL_PASSWORD`, `SURREAL_PASS`). `OPEN_NOTEBOOK_ENCRYPTION_KEY` is separately load-bearing: it is **required to save any provider credential**, and changing it later makes stored keys undecryptable (`:44`).
|
||||
`OPEN_NOTEBOOK_PASSWORD` unset means **authentication off** (`environment-reference.md:46`) — every API route, including the episode-audio download, answers anyone who can reach it. Moving the API to loopback did not remove that exposure, it only removed the LAN from it: any container on `dankclaw_default` dials `open-notebook:5055` by name, so an unset password is an open episode library to every container on this host, guarded or not. That is why `deploy_open_notebook` refuses to start either unit unless the seeded host file carries a value for all four secret keys (`OPEN_NOTEBOOK_ENCRYPTION_KEY`, `OPEN_NOTEBOOK_PASSWORD`, `SURREAL_PASSWORD`, `SURREAL_PASS`). `OPEN_NOTEBOOK_ENCRYPTION_KEY` is separately load-bearing: it is **required to save any provider credential**, and changing it later makes stored keys undecryptable (`:44`).
|
||||
|
||||
The tracked file declares the secrets **empty** and the host file supplies them; podman applies `--env-file` arguments in order and the last one wins (verified against the generated `ExecStart` line, 2026-10-18). Reordering the two `EnvironmentFile=` lines in a unit would blank every credential — that is the trap the seam's ordering assertion exists for.
|
||||
|
||||
@@ -102,7 +134,7 @@ The database has the same trap in a different spelling: the app reads `SURREAL_P
|
||||
|
||||
## Episode destination: interim, and reversible (issue #7)
|
||||
|
||||
**Episode files currently exist in two places by design.** The app's own state stays in the `open-notebook-data` named volume; finished audio additionally lands on the host, because `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` (`open-notebook.container:88`). That is the **interim** answer to where episodes live: the destination is an open decision tracked as **Gitea issue #7, labelled `human-only` — agents must not work it.** Until Frank settles it, episode MP3s are ordinary files in his home directory, readable without `podman unshare`, which the database volume would require (measured with the pinned SurrealDB image).
|
||||
**Episode files currently exist in two places by design.** The app's own state stays in the `open-notebook-data` named volume; everything the render produces additionally lands on the host, because `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` (`open-notebook.container:88`). The first render confirmed the layout: `episodes/<uuid>/clips/0001.mp3 … 00NN.mp3` (one file per dialogue line, written as the worker goes) and `episodes/<uuid>/audio/<uuid>.mp3` (the ffmpeg concatenation). A half-finished render is therefore visible and greppable from the host without `podman unshare`. That is the **interim** answer to where episodes live: the destination is an open decision tracked as **Gitea issue #7, labelled `human-only` — agents must not work it.** Until Frank settles it, episode MP3s are ordinary files in his home directory.
|
||||
|
||||
Reversibility is one line: delete the `Volume=/home/dank/media/podcasts:/app/data/podcasts` line and the episodes go back to living solely inside the named volume. Nothing else in the config assumes the bind. See [[Storage Policy]] for the repo's storage conventions this interim state deliberately bends.
|
||||
|
||||
@@ -118,10 +150,11 @@ podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f7
|
||||
|
||||
## Rollback
|
||||
|
||||
- **Nothing deployed (today's state):** revert the commits. There is no host state to undo — no unit, no container, no volume, no `/home/dank/media/podcasts` directory beyond what the deployer would create.
|
||||
- **Deployed:** `systemctl --user stop open-notebook.service open-notebook-surrealdb.service`, then move the three files (`open-notebook.container`, `open-notebook-surrealdb.container`, `open-notebook.network`) out of `~/.config/containers/systemd/` into `~/.AGENTTRASH/` — never `rm` — and `systemctl --user daemon-reload`. The generated units disappear with their sources.
|
||||
- **Take it off the air, in the reverse of the deploy order:** delete the guarded site block and the `@notebook` handler from `dankclaw/Caddyfile` and deploy `caddy`; then in Authentik delete the `notebook-guard` Application and the `notebook-proxy` provider and drop it from the embedded outpost's provider set. A **route with no provider 502s** while a provider with no route is inert — that asymmetry is why the chain unwinds Caddy-first.
|
||||
- **Stop the app:** `systemctl --user stop open-notebook.service open-notebook-surrealdb.service`, then move the three files (`open-notebook.container`, `open-notebook-surrealdb.container`, `open-notebook.network`) out of `~/.config/containers/systemd/` into `~/.AGENTTRASH/` — never `rm` — and `systemctl --user daemon-reload`. The generated units disappear with their sources.
|
||||
- **Kept on purpose:** the `open-notebook-data` and `open-notebook-db` named volumes and `/home/dank/media/podcasts`. Episode audio and the notebook database are user data; removing them is a separate, deliberate act, and episode files are host files precisely so they can outlive the container.
|
||||
- **Homepage:** the card is `wip: "down"` + `showLink: false`, so retiring the service needs no card surgery beyond setting it back to that state.
|
||||
- **Provider credentials are database rows**, encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`; no repo file mirrors them. Regenerating that key while the database volume survives makes every stored credential undecryptable and forces re-entry — so a rollback that keeps the volume keeps the key file with it.
|
||||
- **Homepage:** the card test in `tests/open_notebook.bats` pins the live state (`showLink: true`, link = the vhost), so retiring the service edits that test in the same commit as the card flip back to `wip: "down"`.
|
||||
|
||||
## Citations
|
||||
|
||||
@@ -130,3 +163,7 @@ podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f7
|
||||
- Registry (2026-10-18): `lfnovo/open_notebook` tags (`v1-latest`, `latest`, the `-single` set) and manifest-list digests; `surrealdb/surrealdb` `v2` → 2.7.0
|
||||
- Local measurements (2026-10-18, workstation, pinned digests): `surreal start --help` (credential and bind options carry `[env: SURREAL_*]`); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, `/surreal is-ready` → `OK` exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two `--env-file` in order, two `--network` in order, named volume + host bind, both health probes, `--pull never`)
|
||||
- Host evidence (Dankclaw, 2026-10-18, read-only): `ss -ltn` (`127.0.0.1:5055` held by seerr, `0.0.0.0:7771` a host listener, nothing on 7772/8502/5056/8000), `podman ps -a` (no open-notebook container, `dankclaw_default` present), subnet registry from `dankclaw/open-webui/quadlet/open-webui.network`
|
||||
- Deploy + provisioning (Dankclaw, 2026-10-19): first-boot log — migrations 1→25, `Credentials were provided, and no root users were found. The root user 'open_notebook' will be created`, `api`/`worker`/`frontend` all RUNNING under supervisord; `GET :8502/config` → the `API_URL` value; `GET /api/providers` (the `openai_compatible` modality list); `GET /openapi.json` schemas for `CreateCredentialRequest` / `ModelCreate` / `SpeakerProfileCreate` / `PodcastGenerateRequest`; the POSTs listed above; the note-save worker error `No embedding model configured.`
|
||||
- First render (2026-10-19): job `command:9rdojq5ne7dj9caicq20`, `execution_time: 301.21 s`, `episode:7ft03thj98sn0ek9xf29`; `episodes/d5fed29b-33e9-40ac-b46a-e0537c56c256/clips/0001.mp3 … 0023.mp3`; `…/audio/d5fed29b-….mp3` = 11,957,420 bytes, mp3 24 kHz mono, `ffprobe` duration 747.31 s; `GET /api/podcasts/episodes/{id}/audio` → 200 `audio/mpeg` with the bearer, 401 without.
|
||||
- Upstream code read inside the pinned image (2026-10-19): `frontend/next.config.ts:26-43` (the `/api/*` rewrite and the one-port claim), `frontend/src/app/config/route.ts:122-173` (`/config` priority order; autodetect appends `:5055`), `frontend/src/app/api/_sse-proxy.ts` (SSE proxied with `no-transform`), `open_notebook/utils/url_validation.py` (link-local refusal, `is_blocked_ip`, no environment override anywhere in the file).
|
||||
- Guard chain (2026-10-19): provider `notebook-proxy` created through `podman exec -i authentik-server ak shell` with the flows copied from `embedded-outpost` (pk 1), Application `Open Notebook`/`notebook-guard`, embedded-outpost provider set 9 → 10; afterwards the outpost logged zero `Failed to fetch outpost from API` and `seerr`/`grafana` still answered 200. Before the route existed, `notebook.franksdank.cloud` resolved to nothing (`000`).
|
||||
|
||||
@@ -111,7 +111,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
||||
|
||||
- Working dir for imports: `/app/backend`; package name `open_webui`
|
||||
(underscore), NOT `openwebui`. `os.environ["WEBUI_SECRET_KEY"]` MUST be set
|
||||
BEFORE importing any `open_webui` module (import-time hard requirement);
|
||||
BEFORE importing any `open_webui` module (import-time hard requirement)
|
||||
the secret lives at `/app/backend/.webui_secret_key`.
|
||||
- **The `sqlite3` CLI is NOT in the container** — use Python's `sqlite3`
|
||||
module (`podman exec open-webui python3 -c "import sqlite3;
|
||||
@@ -122,8 +122,8 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
||||
pydantic `ModelModel` instances (attribute access, not `.get()`), with NO
|
||||
`status`/`enabled` attribute — use `is_active` from the DB.
|
||||
- **API auth for verification:** `create_token` lives in
|
||||
`open_webui.utils.auth` (NOT `open_webui.utils.utils`);
|
||||
`get_verified_user` requires an `id` claim (not just `sub`);
|
||||
`open_webui.utils.auth` (NOT `open_webui.utils.utils`)
|
||||
`get_verified_user` requires an `id` claim (not just `sub`)
|
||||
`SESSION_SECRET = WEBUI_SECRET_KEY`. Mint an admin token inside the
|
||||
container:
|
||||
|
||||
@@ -148,7 +148,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
||||
`ClientConnectorError`: a network-reachability problem (host LAN IP vs
|
||||
`host.containers.internal`), not a model problem.
|
||||
|
||||
**Durability:** the model table + connections are DB-only (not git-managed);
|
||||
**Durability:** the model table + connections are DB-only (not git-managed)
|
||||
a fresh data volume loses them — including the public access grant above.
|
||||
For reproducibility, add an idempotent first-start seed script or document
|
||||
the setup in the service doc.
|
||||
@@ -156,7 +156,7 @@ the setup in the service doc.
|
||||
## API access (auth-gated)
|
||||
|
||||
Every `/api/*` endpoint requires an authenticated session (SSO login JWT)
|
||||
or a per-user API key (`auth.enable_api_keys=true`; keys are minted per
|
||||
or a per-user API key (`auth.enable_api_keys=true`; keys are minted
|
||||
account under Profile → API Keys). Anonymous web visitors get the `pending`
|
||||
role (`ui.default_user_role=pending`), which cannot chat. Verified
|
||||
2026-10-07 from loopback `:3005` and over the public vhost:
|
||||
@@ -166,13 +166,12 @@ bogus bearer token); only `/api/config`, `/api/version` and `/health`
|
||||
`POST /api/chat/completions` all return **401** with no key (also with a bogus bearer
|
||||
token); only `/api/config`, `/api/version` and `/health` answer publicly, and none of
|
||||
them expose the model list. An external program needs an account + API key — there is
|
||||
no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`,
|
||||
no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`
|
||||
`/api/version` → 0.11.4, `/api/v1/models/list`, `/api/v1/models` and
|
||||
`POST /api/chat/completions` → 401; `/api/config` additionally publishes
|
||||
`features.enable_login_form=false`, `features.enable_signup=false` and
|
||||
`oauth.providers.oidc="authentik"` — the SSO-only posture, visible from outside.
|
||||
|
||||
|
||||
## Web search
|
||||
|
||||
Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
|
||||
@@ -186,7 +185,7 @@ Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
|
||||
The Anubis→Authentik guard chain is a browser mechanism (proof-of-work +
|
||||
session flow); a plain HTTP client cannot solve it, and the internal port
|
||||
bypasses Caddy entirely.
|
||||
- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`,
|
||||
- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`
|
||||
a hardcoded Python constant — no config knob). The `pass_ip` list is checked
|
||||
FIRST and short-circuits the whole botdetection chain, and OpenWebUI's bridge
|
||||
egress (`10.89.x`) is in `pass_ip`, so the cap never applies. Verified: an
|
||||
@@ -249,7 +248,6 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
|
||||
restore the password form: set `ui.enable_login_form` back to `true` in the
|
||||
Admin Panel (or DB) and drop the two env vars from the quadlet.
|
||||
|
||||
|
||||
## Boot persistence (Quadlet)
|
||||
|
||||
The unit lives at `dankclaw/open-webui/quadlet/open-webui.container`. On the host it installs to `~/.config/containers/systemd/open-webui.container`. The generator turns it into `open-webui.service`. Because the unit sets `[Install] WantedBy=default.target` and user linger is on, it starts at boot. `systemctl --user is-enabled open-webui.service` reports `generated` (the quadlet boot-start state).
|
||||
@@ -296,7 +294,7 @@ TTS is the fleet's one OpenAI-compatible speech endpoint —
|
||||
`http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
|
||||
in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
|
||||
port is the rootless quadlet `frank-tts` (this repo:
|
||||
`dankclaw/voiceservice/quadlet/frank-tts.container`,
|
||||
`dankclaw/voiceservice/quadlet/frank-tts.container`
|
||||
`scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
|
||||
`/home/dank/voices/<name>/reference.wav` — a name without a reference is a
|
||||
400 listing the voices that exist, never a silent fallback.
|
||||
@@ -310,7 +308,7 @@ setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
|
||||
`browser-kokoro` is chosen the page synthesizes locally and rejects any
|
||||
voice outside its fixed pack — so `narrator1` fails with
|
||||
`Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
|
||||
single request reaching the server**. Personal audio settings are per
|
||||
single request reaching the server**. Personal audio settings are
|
||||
browser: no `user.settings` row and no container env carries an audio
|
||||
override, yet the toast still appears ⇒ the choice lives in that browser's
|
||||
own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
|
||||
@@ -325,7 +323,6 @@ the failing click — that contrast is the instrument: a server-path failure
|
||||
leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
|
||||
neither.
|
||||
|
||||
|
||||
## Retired instances
|
||||
|
||||
- The compose `open-webui` service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
|
||||
@@ -355,7 +352,7 @@ neither.
|
||||
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
||||
id-keyed objects had to move in the same window as the alias flip:
|
||||
`model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
|
||||
`{"context":{"length":200704}}` and the `meta.description` kept),
|
||||
`{"context":{"length":200704}}` and the `meta.description` kept)
|
||||
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
||||
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
||||
`model_ids` (index 2 = `:7771` is the enabled family backend; indices 0/1
|
||||
|
||||
@@ -88,7 +88,7 @@ never enter this repo. See [[Gitea]] and [[Authentik]].
|
||||
|
||||
## Boot persistence (Quadlet)
|
||||
|
||||
Both units carry `Restart=always` (`gitea-postgres.container:28`,
|
||||
Both units carry `Restart=always` (`gitea-postgres.container:28`
|
||||
`authentik-postgres.container:38`) and `[Install] WantedBy=default.target`
|
||||
(`gitea-postgres.container:31`, `authentik-postgres.container:41-42`), are installed
|
||||
under `~/.config/containers/systemd/`, and report `generated` / `active` with
|
||||
|
||||
@@ -143,7 +143,7 @@ engines:
|
||||
disabled: false
|
||||
```
|
||||
|
||||
Key per-engine keys (each overwrites the engine-module default; see
|
||||
Key per-engine keys (each overwrites the engine-module default
|
||||
`settings_engines.html`):
|
||||
|
||||
| Key | Effect |
|
||||
@@ -164,7 +164,7 @@ Key per-engine keys (each overwrites the engine-module default; see
|
||||
*different* category. When the primary `google` gets CAPTCHA'd it suspends, but
|
||||
`google2` still returns results, and vice-versa.
|
||||
|
||||
**Parser caveat (read before fiddling with `parsers`):** the modern `google`,
|
||||
**Parser caveat (read before fiddling with `parsers`):** the modern `google`
|
||||
`duckduckgo`, and `brave` engines ship a **custom `response()`** with hardcoded
|
||||
XPath and *ignore* the generic `parsers`/`p_parsers` table. So `parsers:
|
||||
[scrapy_html]` under `name: google` mostly does nothing. To make Google
|
||||
@@ -376,14 +376,14 @@ done
|
||||
|
||||
**2. `/config` REST endpoint** lists every engine + enabled/disabled +
|
||||
categories — an inventory of what is live. It answers 200 on this instance today
|
||||
(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`,
|
||||
(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`
|
||||
`categories`, `timeout`, `shortcut`, and it is the source of the engine counts on
|
||||
[[SearXNG]]). `?engines=<name>` isolates a single
|
||||
provider; `/debug` adds engine diagnostics.
|
||||
|
||||
**3. `check.searx.space`** — the hosted official checker probes every engine of
|
||||
your instance with a **headless Firefox** (catches Cloudflare/CAPTCHA blocks a
|
||||
plain HTTP probe misses) and posts results on searx.space. Add your instance via
|
||||
plain HTTP probe misses) and posts results on searx.space. Add your instance
|
||||
a GitHub issue in `searxng/searx-instances`, then allow `check.searx.space` IPs
|
||||
to reach the SearXNG port.
|
||||
|
||||
@@ -417,10 +417,10 @@ misleading).
|
||||
already replaced — checked 2026-10-18; see [[SearXNG]]). It is
|
||||
also immune to the shared-VPN-exit-IP blocking this page is about: it
|
||||
authenticates, so reputation of the exit IP is irrelevant.
|
||||
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
|
||||
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to
|
||||
exactly which engines are down and why. Confirm the running SearXNG version
|
||||
(check whether `parsers`/`rate_limit` exist in the pinned build).
|
||||
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds per
|
||||
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds
|
||||
upstream request (`shared/searxng/settings.yml:26`), caps it at 15 seconds
|
||||
(`:27`), and retries once (`:32`). This addresses
|
||||
modest VPN latency without multiplying 429/CAPTCHA traffic. Adjust only
|
||||
|
||||
@@ -108,7 +108,7 @@ engines:
|
||||
disabled: true
|
||||
```
|
||||
|
||||
The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas);
|
||||
The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas)
|
||||
the deployed render at `/opt/searxng/config/settings.yml` differs only in the baked
|
||||
`secret_key`, `base_url` and `KAGI_API_KEY` values, and the running engine set is
|
||||
censused below.
|
||||
@@ -154,7 +154,7 @@ what keeps the `trusted_proxies` entry meaningful.
|
||||
|
||||
## Unblocking Google & Search Providers
|
||||
|
||||
The current request policy allows a modest amount of VPN latency (`5.0s` per
|
||||
The current request policy allows a modest amount of VPN latency (`5.0s`
|
||||
request, `15.0s` maximum — `shared/searxng/settings.yml:26-27`) while limiting
|
||||
retries to one (`:32`). This can recover slow
|
||||
providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared
|
||||
@@ -169,10 +169,10 @@ upstream request volume. SearXNG **suspends** offending engines — which is wha
|
||||
up as "Google went unavailable". The default suspension is shorter than the folklore:
|
||||
see the table in [[SearXNG Providers]]. This is upstream, not local.
|
||||
|
||||
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
|
||||
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`
|
||||
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
|
||||
fresh `name` so a CAPTCHA ban on one leaves the other returning results, and —
|
||||
if Google still re-blocks — route the Google engines at a clean-IP proxy via
|
||||
if Google still re-blocks — route the Google engines at a clean-IP proxy
|
||||
per-engine `proxies` in `settings.yml`. See [[SearXNG Providers]] for the full
|
||||
root cause, the suspend-model table, the `engines:` config, and the
|
||||
`/json` verification recipe.
|
||||
@@ -212,17 +212,16 @@ root cause, the suspend-model table, the `engines:` config, and the
|
||||
(`searxng.container:21`), and that is one of the variables SearXNG substitutes at
|
||||
load, so both paths carry the same LAN-IP value. Flagged, not fixed here: it is a
|
||||
config change, out of scope for docs.
|
||||
- Keep the valkey cache container on the network SearXNG can actually reach — see
|
||||
- Keep the valkey cache container on the network SearXNG can actually reach —
|
||||
[[Valkey]] Gotchas: SearXNG itself has no network of its own (it borrows Gluetun's
|
||||
netns), so the network that matters is the one Gluetun and `searxng-valkey` share.
|
||||
|
||||
|
||||
## Kagi (paid API engine)
|
||||
|
||||
Added 2026-09-26 (user-ordered) and still enabled: the running build ships a native
|
||||
`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`),
|
||||
`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`)
|
||||
which sets `Authorization: Bearer <key>` (`kagi.py:103`) and issues
|
||||
`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`),
|
||||
`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`)
|
||||
and declares `require_api_key: True` (`kagi.py:63`).
|
||||
|
||||
- **Enabled**: `weight: 2` alongside bing/duckduckgo, categories
|
||||
@@ -231,10 +230,10 @@ and declares `require_api_key: True` (`kagi.py:63`).
|
||||
`enabled: true`, categories `general, web` (2026-10-18).
|
||||
- **Cost control**: `paging: false` — exactly one API credit per aggregated
|
||||
query; `timeout: 8.0` so a slow paid query is not wasted (scrapers use 5.0)
|
||||
(`shared/searxng/settings.yml:74-77`; upstream paging default is `True`,
|
||||
(`shared/searxng/settings.yml:74-77`; upstream paging default is `True`
|
||||
`kagi.py:67`, so the `false` here is a real override).
|
||||
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`;
|
||||
`deploy-remote.sh` bakes the real key into the live settings.yml via
|
||||
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`
|
||||
`deploy-remote.sh` bakes the real key into the live settings.yml
|
||||
`scripts/searxng-render-settings.py`, reading `/opt/searxng/.env`
|
||||
(owner `dank`, mode 0600 — `ls -l /opt/searxng/.env` 2026-10-18; its key names are
|
||||
the WireGuard set plus `SEARXNG_SECRET`, `SEARXNG_BASE_URL`, `KAGI_API_KEY`).
|
||||
@@ -262,15 +261,15 @@ and declares `require_api_key: True` (`kagi.py:63`).
|
||||
|
||||
- 262 engines in the build, **75 enabled**, categories of the enabled set led by
|
||||
`other` (12), `it` (11), `general` (9), `videos` (8), `stock images` (7).
|
||||
- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`,
|
||||
`github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`,
|
||||
`arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`,
|
||||
`wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`,
|
||||
- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`
|
||||
`github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`
|
||||
`arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`
|
||||
`wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`
|
||||
`youtube`, `vimeo`, `dailymotion`, `bing videos`.
|
||||
- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`,
|
||||
- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`
|
||||
`brave`, `brave.images`, `brave.news`, `brave.videos`, `qwant`.
|
||||
- The three `startpage` stanzas in `shared/searxng/settings.yml:113-124` target
|
||||
engines that **no longer exist in this build** — `/config` lists no `startpage`,
|
||||
engines that **no longer exist in this build** — `/config` lists no `startpage`
|
||||
`startpage images` or `startpage news`, so those overrides are no-ops kept as
|
||||
intent. *(labelled 2026-10-18)*
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
|
||||
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
|
||||
tags:
|
||||
- dankclaw
|
||||
- seerr
|
||||
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
|
||||
authentik's own user UI instead of Seerr. Both backfilled live: the three
|
||||
standard flows copied from a known-good provider, plus an application
|
||||
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
|
||||
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
|
||||
provisioning rules this incident established.
|
||||
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
|
||||
two provisioning rules this incident established.
|
||||
|
||||
## *arr wiring
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Dankclaw Swap Configuration
|
||||
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
|
||||
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
|
||||
tags:
|
||||
- dankclaw
|
||||
- host
|
||||
@@ -144,8 +144,8 @@ frank-dragoon `health=200` throughout.
|
||||
The previous 8 GiB `/swap.img` (dated 2026-07-21) was staged, not deleted, at
|
||||
`dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/swap.img` (8.1 GiB). **That staging
|
||||
directory no longer exists** — `ls /home/dank/.AGENTTRASH` on 2026-10-18 lists only
|
||||
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`,
|
||||
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`,
|
||||
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`
|
||||
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`
|
||||
`20261017-prism-build`, `20261017-v2abl-trial`, and there is no `/root/.AGENTTRASH`.
|
||||
The old swapfile has therefore been purged; nothing on the host still holds it.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
|
||||
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
|
||||
tags:
|
||||
- dankclaw
|
||||
- frank-tts
|
||||
@@ -23,7 +23,7 @@ voice is how a wrong-voice endpoint stays wrong for weeks.
|
||||
state that stays out of the repo, mounted read-only at `/voices` by the façade
|
||||
unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in
|
||||
`scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:**
|
||||
arm A `qwentts.cpp`, decided in [[Local TTS and Voice Cloning]].
|
||||
arm A `qwentts.cpp`, decided in .
|
||||
|
||||
Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade
|
||||
container runs as uid 10000 and reads the mount from outside its own uid map.
|
||||
@@ -137,7 +137,7 @@ invariant to every crop in the tracked pack list.
|
||||
1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop
|
||||
offset is a claim about one exact file, so the digest goes in the env with it.
|
||||
2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once:
|
||||
`ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`,
|
||||
`ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`
|
||||
build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest
|
||||
on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the
|
||||
`.rvq` falls under the floor.
|
||||
@@ -188,15 +188,15 @@ loud, not a bug. Work through it in order:
|
||||
1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec`
|
||||
prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , …
|
||||
codebooks @ … Hz` on every run, and the pinned tree's
|
||||
`tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`;
|
||||
`tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`
|
||||
`codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`.
|
||||
2. Decide whether the new number is intended. A changed `enc_dim` means the old
|
||||
`.spk` files are unreadable by the new engine, so **every** pack must be
|
||||
rebuilt, not just the failing one.
|
||||
3. Update the four arithmetic lines in `scripts/voice-packs.env`
|
||||
(`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`,
|
||||
(`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`
|
||||
and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and**
|
||||
`VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`,
|
||||
`VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`
|
||||
`VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes
|
||||
without moving the commit is the mistake to avoid.
|
||||
4. Rebuild all three packs with `--force` (the retired directories keep the old
|
||||
@@ -226,7 +226,7 @@ loud, not a bug. Work through it in order:
|
||||
narrator1 and narrator2). The builder's first real run is therefore a
|
||||
migration, and it should be a `--force` run per voice so the old directories
|
||||
are retired rather than left half-updated.
|
||||
- The three source files (`/home/blank/audiosample.mp3`,
|
||||
- The three source files (`/home/blank/audiosample.mp3`
|
||||
`audiosample2.m4b`, `audiosample3.m4b`, digests pinned in
|
||||
`scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the
|
||||
builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so
|
||||
|
||||
@@ -80,7 +80,7 @@ image's own nginx (the desktop), so the content server runs on internal 8090.
|
||||
**Auth model (calibre 9.x):** users live in a SQLite userdb whose `pw` column
|
||||
is **plaintext** — the file is mode 600. The single admin account is
|
||||
bootstrapped once by the deploy handler
|
||||
(`deploy_calibre_server` in `scripts/deploy-remote.sh`) via
|
||||
(`deploy_calibre_server` in `scripts/deploy-remote.sh`)
|
||||
`calibre-server --manage-users -- add`, the password piped on stdin with
|
||||
`printf` (a trailing newline from `echo` lands in the stored password and
|
||||
breaks every login). The desktop's basic auth (`CUSTOM_USER`/`CUSTOM_PASSWORD`)
|
||||
|
||||
@@ -83,7 +83,7 @@ The `iv` edition is no longer available for arm32/armv6. Use `dj` for maximum co
|
||||
|
||||
## Volumes
|
||||
|
||||
Live deployment: the whole LUKS array is served at URL `/w` via
|
||||
Live deployment: the whole LUKS array is served at URL `/w`
|
||||
`Volume=/mnt/entropy:/w:ro` in the quadlet unit ([[Entropy Volume]]) —
|
||||
read-only **at the mount level**, so no containerized consumer can write into
|
||||
the array through this path regardless of account permissions. The ac edition
|
||||
@@ -111,7 +111,7 @@ convention; see [[#Command-line interface (ac edition)]] above.
|
||||
```
|
||||
|
||||
`${COPAY_ACCOUNT}`/`${COPAY_PASSWORD}` are **never** written into the unit:
|
||||
they live in the host-local `/home/dank/Containers/copyparty/env` (0600),
|
||||
they live in the host-local `/home/dank/Containers/copyparty/env` (0600)
|
||||
loaded via `[Service] EnvironmentFile=` so systemd expands them into
|
||||
`Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD}` at start. The password still
|
||||
lands in the container argv — visible to anyone who can run
|
||||
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
|
||||
|
||||
## Security Advisories
|
||||
|
||||
| Version | CVE | Issue |
|
||||
|---------|-----|-------|
|
||||
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
|
||||
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
|
||||
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
|
||||
| Version | Advisory | CVE | Issue |
|
||||
|---------|----------|-----|-------|
|
||||
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
|
||||
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
|
||||
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
|
||||
|
||||
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
|
||||
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
|
||||
- [[Dankserver Stack]]
|
||||
- [[Caddy]] — TLS termination and authentication in front
|
||||
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
|
||||
|
||||
## Citations
|
||||
|
||||
- GitHub Security Advisories, `9001/copyparty` —
|
||||
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19)
|
||||
each row's patched version read from `first_patched_version` in the
|
||||
`https://api.github.com/advisories/<GHSA-id>` response.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Dankserver (192.168.0.235) runs the media and library stack."
|
||||
description: "Service inventory and routing overview for the media and library host."
|
||||
tags:
|
||||
- dankserver
|
||||
timestamp: 2026-09-03
|
||||
@@ -54,5 +54,5 @@ The following services ran on Dankserver and are now retired from this stack:
|
||||
|
||||
## See Also
|
||||
|
||||
- [[index]] — repository index
|
||||
- repository index
|
||||
- [[Network Architecture]] — routing and proxy
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Jellyfin
|
||||
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
|
||||
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
|
||||
tags: [jellyfin, dankserver, media-server]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
@@ -14,7 +14,7 @@ serving music, movies, and shows over LAN. It replaced the earlier research
|
||||
brief; it runs alongside [[Plex]], which remains the primary server for some
|
||||
clients.
|
||||
|
||||
**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`,
|
||||
**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`
|
||||
public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
[[Authentik]] outpost (post-auth route proxies `localhost:8096`).
|
||||
|
||||
@@ -67,7 +67,7 @@ user to verify the password.
|
||||
- LDAP provider `Jellyfin LDAP` (Base DN `dc=franksdank,dc=cloud`, bind/search
|
||||
mode `direct`) + application `Jellyfin`.
|
||||
- LDAP outpost `Jellyfin LDAP Outpost` → container `authentik-outpost-ldap`
|
||||
(quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`),
|
||||
(quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`)
|
||||
publishing `389`/`636` on the Dankclaw host for cross-host binds.
|
||||
- Service account `svc-jellyfin-ldap` (DN
|
||||
`cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`).
|
||||
@@ -80,7 +80,7 @@ user to verify the password.
|
||||
**Jellyfin side** (plugin config, set via admin → Plugins → LDAP-Auth):
|
||||
- Server `192.168.0.143:389` (plain LDAP over the trusted LAN), Base DN
|
||||
`dc=franksdank,dc=cloud`, bind user = the service-account DN, search
|
||||
attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`,
|
||||
attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`
|
||||
`CreateUsersFromLdap=true`, all folders enabled.
|
||||
|
||||
**Gotcha — the bind credential is a PASSWORD, not a token.** The LDAP bind is
|
||||
|
||||
@@ -60,7 +60,7 @@ root runs as the `dank` user, and the deployer adopts the data root with
|
||||
## Integration points
|
||||
|
||||
- **Caddy:** Plex is exempt from the Anubis bot challenge
|
||||
(plain-HTTP legacy clients cannot solve JS challenges) — see
|
||||
(plain-HTTP legacy clients cannot solve JS challenges) —
|
||||
[[Anubis]] "Exempt by construction".
|
||||
- **Tautulli** ([[Tautulli]]) is the companion: it polls PMS on the appnet
|
||||
for statistics and notifications and makes no sense without Plex.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
|
||||
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
|
||||
tags:
|
||||
- dankserver
|
||||
- romm
|
||||
|
||||
@@ -32,7 +32,7 @@ Anubis → Authentik chain ([[Caddy]] routes it post-auth to
|
||||
## Architecture
|
||||
|
||||
Current RomM releases require MariaDB plus Redis (or Valkey) for the task
|
||||
queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit),
|
||||
queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit)
|
||||
not valkey.
|
||||
|
||||
```mermaid
|
||||
@@ -114,7 +114,7 @@ scan:
|
||||
## Deployment (Quadlet)
|
||||
|
||||
Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
|
||||
`./scripts/deploy.sh dankserver --service romm` (any of `romm`,
|
||||
`./scripts/deploy.sh dankserver --service romm` (any of `romm`
|
||||
`romm-mariadb`, `romm-redis` triggers the full ordered cutover):
|
||||
|
||||
- Units: `dankserver/romm/quadlet/{romm,romm-mariadb,romm-redis}.container`
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: MariaDB (RomM Backend)
|
||||
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
|
||||
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
|
||||
tags: [mariadb, romm, dankserver]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Redis (RomM Cache / Queue)
|
||||
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
|
||||
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
|
||||
tags: [redis, romm, dankserver]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Noctalia and Niri
|
||||
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
|
||||
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
|
||||
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
@@ -45,14 +45,14 @@ Verified: `msg status` returns live JSON state; unknown panel ids return an
|
||||
error listing all valid panel ids (clipboard, control-center, launcher,
|
||||
polkit, session, setup-wizard, test, tray-drawer, wallpaper).
|
||||
|
||||
Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`,
|
||||
`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`,
|
||||
`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`,
|
||||
Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`
|
||||
`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`
|
||||
`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`
|
||||
`media next|previous|playPause` (graceful no-op, rc 0, with no active
|
||||
player), `brightness-up [target] [step]`, `brightness-down`,
|
||||
`settings-open [context]`, `settings-toggle [context]`, `config-reload`,
|
||||
`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`,
|
||||
`color-scheme-get|set`, `theme-mode-get`, `templates-apply`,
|
||||
player), `brightness-up [target] [step]`, `brightness-down`
|
||||
`settings-open [context]`, `settings-toggle [context]`, `config-reload`
|
||||
`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`
|
||||
`color-scheme-get|set`, `theme-mode-get`, `templates-apply`
|
||||
`screenshot-fullscreen`.
|
||||
|
||||
**Wallpaper directory semantics:** empty/unset `[wallpaper] directory`
|
||||
@@ -81,9 +81,9 @@ Custom palettes live at `~/.config/noctalia/palettes/<Name>.json`
|
||||
synced. Missing/invalid file → silent fallback to the builtin palette.
|
||||
|
||||
Palette JSON shape = the community file's shape: `dark`/`light` maps of
|
||||
`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`,
|
||||
`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`
|
||||
`mSurfaceVariant`, `mOutline`, `mShadow`, `mHover` plus a `terminal` block
|
||||
(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`,
|
||||
(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`
|
||||
`cursor`, `cursorText`, `selectionFg`, `selectionBg`). NOTE: `noctalia theme
|
||||
-o` emits M3 snake_case tokens (`on_primary_container`, …) — that is the
|
||||
*generator* format, not this one. Copy an existing community JSON and mutate
|
||||
@@ -114,7 +114,7 @@ Procedure:
|
||||
`noctalia msg color-scheme-set custom "Gruber Red"` → `ok`; confirm with
|
||||
`color-scheme-get`.
|
||||
5. Re-render every configured template: `noctalia msg templates-apply`.
|
||||
Regenerates `~/.config/ghostty/themes/noctalia`,
|
||||
Regenerates `~/.config/ghostty/themes/noctalia`
|
||||
`~/.config/gtk-{3,4}.0/noctalia.css`, kcolorscheme, foot, wezterm,
|
||||
starship, etc. Symlinked-into-dotfiles paths show up as modified files
|
||||
worth committing.
|
||||
@@ -216,7 +216,7 @@ Verified on blankpad, niri 26.04 (two PM1601 externals + eDP-1 laptop panel).
|
||||
scale, transform, logical pos/size) and `niri --version`. Output **names are
|
||||
connector names** (`eDP-1`, `DP-2`, `HDMI-A-1`), not EDID model strings —
|
||||
the EDID strings printed before them are decoration. On blankpad the live
|
||||
config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`,
|
||||
config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`
|
||||
and `cfg/display.kdl` is a **symlink into the dotfiles repo** — edit the
|
||||
repo path (that is what gets committed); check with `ls -ld`.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "*arr Stack"
|
||||
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
|
||||
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
|
||||
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
|
||||
timestamp: 2026-09-04T00:00:00Z
|
||||
---
|
||||
@@ -38,7 +38,7 @@ live call at a time instead of guessing payload shapes:
|
||||
- `'Base Url' must not be empty` when keys were camelCase → **PascalCase is
|
||||
required** (`BaseUrl`, `ApiKey`) — schema keys, not the JSON wire keys.
|
||||
- `DefaultMetadataProfileId must be > 0` → fetch real ids from
|
||||
`GET /api/v1/metadataprofile` (rows like `(1,'Standard')`);
|
||||
`GET /api/v1/metadataprofile` (rows like `(1,'Standard')`)
|
||||
`GET /api/v1/qualitydefinition` returns the quality CATALOG — ids ≠
|
||||
profiles.
|
||||
- `targetType null` → configContract not registered in that build (this
|
||||
@@ -87,7 +87,7 @@ settings values live in `prowlarr.db` → `Applications.Settings` JSON (the
|
||||
pipeline hides a miss and you conclude "empty field" when the grep itself
|
||||
failed.
|
||||
4. Decision framing: when posture = gated-with-zero-users, present it as a
|
||||
decision menu (None behind the front-door wall vs External/authentikarr),
|
||||
decision menu (None behind the front-door wall vs External/authentikarr)
|
||||
not a silent change — first-account-wins means whoever logs in first owns
|
||||
the instance.
|
||||
|
||||
|
||||
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
|
||||
|
||||
## *arr Stack
|
||||
|
||||
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
|
||||
- [[sonarr]] — TV show management
|
||||
- [[radarr]] — movie management
|
||||
- [[lidarr]] — music management
|
||||
- [[readarr]] — ebook/audiobook management
|
||||
- [[prowlarr]] — indexer management
|
||||
- [[qbittorrent]] — torrent download client
|
||||
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
|
||||
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
|
||||
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
|
||||
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
|
||||
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Prowlarr
|
||||
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
|
||||
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
|
||||
tags: [prowlarr, arr-stack, frankbook-pro]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
@@ -70,7 +70,7 @@ http://prowlarr:9696/{indexerId}/download?link=...&file=... (torrent handoff)
|
||||
The global API key is in `config.xml` (`<ApiKey>`); per-application keys are
|
||||
only used by Prowlarr's own sync calls, NOT by the feeds. The arr download
|
||||
clients authenticate to qBittorrent with the permanent WebUI password stored
|
||||
in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver — see
|
||||
in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver —
|
||||
[[qBittorrent]] for why that password must be pinned in the conf file.
|
||||
|
||||
**Readarr caveat:** Readarr is abandoned upstream; its qBittorrent client
|
||||
@@ -97,13 +97,13 @@ successor; the indexer row itself is healthy.
|
||||
- Categories in the indexer row must be a subset of what the feed's `caps`
|
||||
advertises; extras log "does not support the following categories" and
|
||||
some feeds reject them.
|
||||
- **A freshly created indexer row defaults to all-off.** If `enableRss`,
|
||||
- **A freshly created indexer row defaults to all-off.** If `enableRss`
|
||||
`enableAutomaticSearch`, or `enableInteractiveSearch` aren't set, the
|
||||
app logs `Searching indexers for [X]. 0 active indexers` (Lidarr,
|
||||
verified). Search proves TV + music chains 2026-08-30: Sonarr returned
|
||||
76 live releases for an anime term, Lidarr 18 for YOASOBI – THE BOOK,
|
||||
all via the Prowlarr-Nyaa feed.
|
||||
- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`);
|
||||
- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`)
|
||||
GET responses carry `fields: []` — real settings values live in
|
||||
`prowlarr.db` → `Applications.Settings` JSON (read-only inspection).
|
||||
- Instance name/`InstanceName=Prowlarr` in `config.xml` does NOT prefix the
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: qBittorrent
|
||||
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
|
||||
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
|
||||
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
@@ -13,7 +13,7 @@ qBittorrent is the torrent download client for the *arr stack. The stack runs on
|
||||
not on Dankserver. It receives download requests from Sonarr, Radarr, Lidarr,
|
||||
and Readarr via category tagging.
|
||||
|
||||
**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`,
|
||||
**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`
|
||||
published by the [[Gluetun]] unit because qBittorrent shares that container's
|
||||
network namespace (a netns-sharer publishes no ports of its own).
|
||||
|
||||
@@ -121,7 +121,7 @@ Mac shell export `PATH="/usr/local/bin:/opt/homebrew/bin:$PATH"` for
|
||||
`podman machine ssh`.
|
||||
|
||||
**Access note (2026-09-21):** the Mac's login user is **dank**@192.168.0.112
|
||||
(NOT core — `core` is the podman-machine VM user, reached only via
|
||||
(NOT core — `core` is the podman-machine VM user, reached only
|
||||
`podman machine ssh`). blankpad has key auth for `dank@` (bootstrapped
|
||||
2026-09-21) and a `frankbook-pro` alias in `~/.ssh/config`. macOS sshd has a
|
||||
low MaxAuthTries: when password-bootstrapping use `-o
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
---
|
||||
type: reference
|
||||
title: Readarr
|
||||
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
|
||||
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
|
||||
tags: [readarr, arr-stack, frankbook-pro]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
---
|
||||
|
||||
# Readarr
|
||||
|
||||
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
|
||||
> metadata became unusable and the community migration to Open Library stalled.
|
||||
> There is no upstream development. Alternatives: the Bookshelf community fork,
|
||||
> or the Calibre content server. This instance still runs on a pinned image
|
||||
> tag; see below.
|
||||
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
|
||||
> Readarr: its metadata became unusable and the community migration to Open
|
||||
> Library stalled. There is no upstream development — the GitHub repository is
|
||||
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
|
||||
> community fork, or the Calibre content server. This instance still runs on a
|
||||
> pinned image tag; see below.
|
||||
|
||||
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
|
||||
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
|
||||
@@ -69,9 +70,9 @@ Restart=always
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
|
||||
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
|
||||
`:latest` does not exist for this image on LSIO.
|
||||
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
|
||||
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
|
||||
because `:latest` does not exist for this image on LSIO.
|
||||
- Calibre integration is folder-based, not API-based. Readarr drops files into
|
||||
the books root which [[Calibre-Web]] serves.
|
||||
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
|
||||
@@ -81,3 +82,10 @@ Restart=always
|
||||
|
||||
- [[ARR-Stack]] — full *arr stack documentation
|
||||
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
|
||||
|
||||
## Citations
|
||||
|
||||
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
|
||||
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
|
||||
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
|
||||
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
|
||||
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
|
||||
tags:
|
||||
- infrastructure
|
||||
timestamp: 2026-09-17
|
||||
|
||||
@@ -93,7 +93,7 @@ and `hermesadmin.` (app-enforced passwords, no public DNS for the first,
|
||||
(`dankclaw/Caddyfile:241-251`), and the FrankBook-Pro *arr/qBittorrent vhosts
|
||||
(`dankclaw/Caddyfile:260-352`). There is also no vhost at all for ComfyUI, so
|
||||
it is exempt by absence — and therefore unauthenticated on the LAN at
|
||||
`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`,
|
||||
`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`
|
||||
`grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`
|
||||
(`dankclaw/Caddyfile:283-423`).
|
||||
|
||||
@@ -140,7 +140,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
|
||||
instead of allowed, the browser's fetch throws and it renders its own
|
||||
error page (iOS/macOS browsers phrase it as "the request failed and the
|
||||
interceptors did not return an alternative response") instead of the
|
||||
login flow. Verify with a JS-capable client end-to-end — mint via
|
||||
login flow. Verify with a JS-capable client end-to-end — mint
|
||||
`make-challenge`, solve, submit with browser-exact query params — before
|
||||
declaring a challenge chain healthy.
|
||||
7. **Single-file bind mount + git pull = stale policy.** The policy is
|
||||
@@ -158,7 +158,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
|
||||
answers any request lacking `X-Real-Ip` with a 500 whose log line reads
|
||||
`[misconfiguration] X-Real-Ip header is not set`. The real chain always
|
||||
sets it (every guarded Caddy listener carries `header_up X-Real-Ip
|
||||
{remote_host}` — the bats seam enforces the line), so users never see
|
||||
{remote_host}` — the bats seam enforces the line), so users never
|
||||
this. But any direct probe of `:8923` (scratch container, curl, a new
|
||||
reverse-proxy hop) MUST set the header or it looks like a policy failure.
|
||||
Scratch-verify with `-H "X-Real-Ip: 192.168.0.20"`.
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Authentik Outpost Reference
|
||||
|
||||
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
|
||||
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
|
||||
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
|
||||
self-serve user-settings flow, and the live database shape. What the service is,
|
||||
how it is deployed, wired and operated is [[Authentik]]; the three core
|
||||
containers are [[Authentik Processes]].
|
||||
|
||||
## Live proxy-provider inventory
|
||||
|
||||
Live provider inventory (read from the `authentik-postgres` container with
|
||||
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
||||
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
||||
members in `authentik_outposts_outpost_providers`), all with
|
||||
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
||||
UUIDs** (`default-authentication-flow`
|
||||
`default-provider-authorization-implicit-consent`
|
||||
`default-provider-invalidation-flow`), and each backed by an Application
|
||||
(the `slug` column below):
|
||||
|
||||
| Provider pk | name | external_host | app slug |
|
||||
|---|---|---|---|
|
||||
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
||||
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
||||
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
||||
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
||||
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
||||
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
||||
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
||||
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
||||
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
||||
|
||||
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
||||
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
|
||||
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
||||
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
||||
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
||||
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
||||
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
||||
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
||||
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
||||
below rejects), and pk 14 `Open WebUI`
|
||||
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
||||
`external_host` (the runbook-recipe style) and bare origins both intercept
|
||||
correctly — the outpost matches on hostname.
|
||||
|
||||
## Provisioning rules for proxy providers
|
||||
|
||||
Two provisioning rules, each verified live by an outage:
|
||||
|
||||
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
||||
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
||||
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
||||
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
||||
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
||||
forever, and **everything behind the outpost dies at the next outpost
|
||||
reload** — including hosts whose providers were fine (verified 2026-09-02
|
||||
when three providers provisioned without any flows took down every guarded
|
||||
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
||||
`default-authentication-flow`
|
||||
`default-provider-authorization-implicit-consent` (the implicit-consent
|
||||
authorization flow, `default-invalidation-flow`) and the retry loop
|
||||
recovers on its next 3-second cycle. The silent killer: while the outpost
|
||||
process keeps its last good in-memory config, everything works — the
|
||||
outage only surfaces on restart/reload.
|
||||
2. **Every proxy provider needs an attached Application.** The outpost
|
||||
resolves the post-login redirect through the provider's application; a
|
||||
provider with no app completes the login (authentik session established)
|
||||
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
||||
the app. No error is logged anywhere — the SSO "works" and the app never
|
||||
appears. Create the Application with `provider_id` set (the UI does this
|
||||
automatically; raw API/ORM drives must do it explicitly).
|
||||
|
||||
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
||||
|
||||
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
||||
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
||||
implementation diverges from upstream in ways that bit us; all verified live:
|
||||
|
||||
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
||||
create the single-table-inheritance `Application` row; without it the flow
|
||||
executor can never complete an authorize request. Create both rows.
|
||||
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
||||
contains `"authorization_code"` (enum value with underscore), the token
|
||||
endpoint rejects every exchange with `invalid_request`.
|
||||
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
||||
`parse_token_request` see no `grant_type` → `unsupported_grant_type`
|
||||
POST-body client secrets are not accepted — authenticate the confidential
|
||||
client with `Authorization: Basic base64(client_id:client_secret)`.
|
||||
- **Provider resolution is by `client_id` query param** on the global
|
||||
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
||||
this build. Token/userinfo live at `/application/o/token/` and
|
||||
`/application/o/userinfo/?api=1`.
|
||||
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
||||
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
||||
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
||||
becomes the *expression source* instead of the context; every call raises a
|
||||
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
||||
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
||||
provider FK (linkage lives in the `ProviderPropertyMapping` through model)
|
||||
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
||||
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
||||
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
||||
build until the evaluator is patched or upgraded. Clients consuming this
|
||||
provider must key identity off `sub`.
|
||||
- Scope mappings for this provider exist — `grafana-email-scope` and
|
||||
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
||||
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
||||
the evaluator bug is fixed; no config change is needed then.
|
||||
|
||||
## LDAP provider objects (provisioned via `ak shell` ORM)
|
||||
|
||||
The LDAP outpost's own deployment — unit, image digest, published ports, token
|
||||
env var — is in [[Authentik]]. The objects it serves:
|
||||
|
||||
Read from the live database (read-only `SELECT`s against the
|
||||
`authentik-postgres` container, 2026-10-18):
|
||||
|
||||
| Object | Value |
|
||||
|---|---|
|
||||
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
||||
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
||||
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
||||
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
|
||||
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
||||
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
||||
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
|
||||
|
||||
Hard-won rules (the first is now contested by the live database):
|
||||
|
||||
- **Token intent: contested, do not trust either value blindly.** An earlier
|
||||
revision asserted that a bind token with `intent=app_password` is rejected by
|
||||
the flow's password stage with `Invalid password` and only
|
||||
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
|
||||
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
|
||||
wrong or the current Jellyfin bind path is not that token. Nothing observable
|
||||
settles it today: the LDAP outpost's retained log holds only its 5-minute
|
||||
`initialized direct binder` / `Update providers` config cycles and no bind
|
||||
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
||||
requires the bind secret)*. Re-verify with a real bind before changing the
|
||||
token's intent either way.
|
||||
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
||||
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
||||
matches zero entries and the plugin logs `Found no users matching X`.
|
||||
The permission is object-level on the provider; grant it via an RBAC role
|
||||
and retest.
|
||||
- `ak shell` import path for the provider model in 2026.8:
|
||||
`from authentik.providers.ldap.models import LDAPProvider`.
|
||||
- (unverified 2026-10-18: importing it outside a configured Django shell fails
|
||||
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
||||
auth service for this check)*
|
||||
|
||||
## Self-serve user-settings flow (password change)
|
||||
|
||||
Users change their own account details through the `default-user-settings-flow`
|
||||
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
||||
target of the homepage footer's "account" link. It is a `stage_configuration`
|
||||
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
||||
user.
|
||||
|
||||
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
||||
insertion order):
|
||||
|
||||
| order | stage | type | what it does |
|
||||
|---|---|---|---|
|
||||
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
||||
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
||||
|
||||
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
||||
writes whatever the preceding `PromptStage` collected: for the key `password`
|
||||
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
||||
is a `password` field on the `default-user-settings` prompt
|
||||
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
||||
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
||||
`validate_password` runs `authenticate()` against the backends to check the
|
||||
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
||||
2026-09-25 and removed the same day for exactly this reason.)
|
||||
|
||||
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
||||
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
||||
MTI), so create the `FlowStageBinding` row directly with an explicit `order`
|
||||
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
||||
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
||||
|
||||
**A blank password must not set an empty password.** `set_password("")` would
|
||||
make the account log in with no password, so a validation policy
|
||||
(`default-user-settings-no-empty-password`, bound to the prompt
|
||||
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
||||
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
||||
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
||||
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
||||
invalid).
|
||||
|
||||
## Live database shape
|
||||
|
||||
For live DB truth prefer the postgres container, and it works without any
|
||||
credential — the local socket is trusted, so a read-only audit is just
|
||||
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
||||
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
||||
before guessing: 2026.x renamed both tables and columns, and the models are
|
||||
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
||||
three `*_flow_id` columns; the proxy table is
|
||||
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
||||
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
||||
table in 2026.8), the LDAP table is
|
||||
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`
|
||||
`authentik_core_application` attaches to a provider through `provider_id`
|
||||
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
||||
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
||||
is the same story from the ORM side: `type` lives in the subclass tables.
|
||||
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
||||
failure modes
|
||||
- [[Authentik Processes]] — the postgres, server and worker containers
|
||||
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
|
||||
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
|
||||
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
- postgres
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# Authentik Processes
|
||||
|
||||
The three core containers of the [[Authentik]] stack on Dankclaw. The two
|
||||
outpost units are not covered here — they live on [[Authentik]] and
|
||||
[[Authentik Outpost Reference]]. Server and worker run the same image; the
|
||||
`command` selects the role (`server` / `worker`). Data persists under
|
||||
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
|
||||
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
|
||||
standardization; the retired stubs this page replaces recorded the pre-move
|
||||
paths `/home/dank/authentik/data` (server and worker) and
|
||||
`/home/dank/authentik/postgres` (database).
|
||||
|
||||
## authentik-postgres
|
||||
|
||||
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
|
||||
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
|
||||
PostgreSQL in 2025.10 (Redis removed).
|
||||
|
||||
- **Image:** `docker.io/library/postgres:16-alpine`
|
||||
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
|
||||
volumes)
|
||||
- **Health:** `pg_isready -d authentik -U authentik`
|
||||
- **Retired wording:** the stub called this a "Compose service … in
|
||||
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
|
||||
- The side-by-side with `gitea-postgres` (network, published ports, secret
|
||||
source, boot persistence) is owned by [[Postgres]].
|
||||
|
||||
## authentik-server
|
||||
|
||||
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
||||
It also serves the login page at `https://auth.franksdank.cloud`.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
|
||||
(`authentik-server.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
|
||||
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
|
||||
live here
|
||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
||||
worker)
|
||||
- **Health:** recorded by the retired stub as
|
||||
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
|
||||
owns the live probe.
|
||||
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
|
||||
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
|
||||
integration.
|
||||
|
||||
## authentik-worker
|
||||
|
||||
The Authentik worker — background tasks: event processing, provider
|
||||
synchronization, scheduled flows, and certificate renewal.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
|
||||
(`authentik-worker.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
|
||||
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
|
||||
the data dir)
|
||||
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
|
||||
server)
|
||||
- **No port** — the worker has no HTTP endpoint.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [[Authentik]] — what the service is, its deployment, wiring, operations and
|
||||
failure modes
|
||||
- [[Authentik Outpost Reference]] — provider, flow and token configuration
|
||||
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
|
||||
@@ -15,26 +15,33 @@ application-level authentication with its proxy outpost. Not wrapped: Gitea,
|
||||
OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
|
||||
they handle their own session. Grafana is the exception that is both: it is a
|
||||
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
|
||||
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
|
||||
`dankclaw/Caddyfile:374-377`; live provider table below).
|
||||
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`
|
||||
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
|
||||
|
||||
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
|
||||
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
|
||||
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`
|
||||
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`
|
||||
2026-10-18)
|
||||
- **Host:** dankclaw (`192.168.0.143`)
|
||||
- **Name:** `auth.franksdank.cloud`
|
||||
- **Stack: five quadlet units, no compose.** `authentik-postgres`,
|
||||
`authentik-server`, `authentik-worker` (`deploy_authentik`,
|
||||
- **Stack: five quadlet units, no compose.** `authentik-postgres`
|
||||
`authentik-server`, `authentik-worker` (`deploy_authentik`
|
||||
`scripts/deploy-remote.sh:980-1004`) plus two outposts — `authentik-outpost`
|
||||
(the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`,
|
||||
(the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`
|
||||
`deploy_authentik_outpost`, `scripts/deploy-remote.sh:1006-1018`) and
|
||||
`authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`,
|
||||
`authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`
|
||||
`scripts/deploy-remote.sh:1020-1032`). All five report `generated` +
|
||||
`active`, `Restart=always`, `WantedBy=default.target`, with linger on
|
||||
(`systemctl --user`, 2026-10-18). The "3 compose services" wording is the
|
||||
pre-quadlet era, retired.
|
||||
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
|
||||
|
||||
**Split off from this page:** [[Authentik Processes]] owns the per-container
|
||||
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`
|
||||
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
|
||||
the live proxy-provider inventory, the proxy-provider provisioning rules, the
|
||||
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
|
||||
design, and the live database shape.
|
||||
|
||||
## Why Authentik
|
||||
|
||||
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
|
||||
@@ -100,23 +107,10 @@ contract:
|
||||
The chain terminates there — nothing dials back into Authentik from
|
||||
`:8017`.
|
||||
|
||||
## The three compose services
|
||||
## The stack's processes
|
||||
|
||||
| Service | Role | Image |
|
||||
|---|---|---|
|
||||
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
|
||||
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
|
||||
|
||||
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
|
||||
moved to PostgreSQL in 2025.10 (Redis removed).
|
||||
- **Server** and **worker** run the same image; the `command` selects the role
|
||||
(`server` / `worker`).
|
||||
- The worker runs as `0:0` (rootless: container root maps to the host user that
|
||||
owns the data dir).
|
||||
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
|
||||
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
|
||||
in the 2026-09-01 quadlet standardization).
|
||||
Per-container facts — unit path, image and role, port, run-as, data directory,
|
||||
health probe — are owned by [[Authentik Processes]].
|
||||
|
||||
## Caddy integration
|
||||
|
||||
@@ -142,110 +136,15 @@ routing table, and the Authentik proxy-provider list must be the same set of
|
||||
hosts. A guarded host with no provider 404s at login time; a provider with no
|
||||
DNS record breaks off-LAN clients.
|
||||
|
||||
Live provider inventory (read from the `authentik-postgres` container with
|
||||
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
|
||||
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
|
||||
members in `authentik_outposts_outpost_providers`), all with
|
||||
`internal_host = http://caddy:8017`, all carrying the **same three flow
|
||||
UUIDs** (`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent`,
|
||||
`default-provider-invalidation-flow`), and each backed by an Application
|
||||
(the `slug` column below):
|
||||
|
||||
| Provider pk | name | external_host | app slug |
|
||||
|---|---|---|---|
|
||||
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
|
||||
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
|
||||
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
|
||||
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
|
||||
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
|
||||
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
|
||||
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
|
||||
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
|
||||
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
|
||||
|
||||
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
|
||||
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
|
||||
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
|
||||
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
|
||||
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
|
||||
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
|
||||
not proxy providers, so the proxy outpost does not deserialize them: pk 9
|
||||
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
|
||||
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
|
||||
below rejects), and pk 14 `Open WebUI`
|
||||
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
|
||||
`external_host` (the runbook-recipe style) and bare origins both intercept
|
||||
correctly — the outpost matches on hostname.
|
||||
|
||||
Two provisioning rules, each verified live by an outage:
|
||||
|
||||
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
|
||||
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
|
||||
null wedges the embedded outpost's config fetch: the Rust side rejects the
|
||||
whole config JSON (serde: `invalid type: null, expected a formatted UUID
|
||||
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
|
||||
forever, and **everything behind the outpost dies at the next outpost
|
||||
reload** — including hosts whose providers were fine (verified 2026-09-02
|
||||
when three providers provisioned without any flows took down every guarded
|
||||
vhost after a server restart). Backfill all three flows (copy from pk 1:
|
||||
`default-authentication-flow`,
|
||||
`default-provider-authorization-implicit-consent` (the implicit-consent
|
||||
authorization flow, `default-invalidation-flow`) and the retry loop
|
||||
recovers on its next 3-second cycle. The silent killer: while the outpost
|
||||
process keeps its last good in-memory config, everything works — the
|
||||
outage only surfaces on restart/reload.
|
||||
2. **Every proxy provider needs an attached Application.** The outpost
|
||||
resolves the post-login redirect through the provider's application; a
|
||||
provider with no app completes the login (authentik session established)
|
||||
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
|
||||
the app. No error is logged anywhere — the SSO "works" and the app never
|
||||
appears. Create the Application with `provider_id` set (the UI does this
|
||||
automatically; raw API/ORM drives must do it explicitly).
|
||||
|
||||
|
||||
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
|
||||
|
||||
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
|
||||
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
|
||||
implementation diverges from upstream in ways that bit us; all verified live:
|
||||
|
||||
- **Application row required.** `OAuth2Provider.objects.create()` does not
|
||||
create the single-table-inheritance `Application` row; without it the flow
|
||||
executor can never complete an authorize request. Create both rows.
|
||||
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
|
||||
contains `"authorization_code"` (enum value with underscore), the token
|
||||
endpoint rejects every exchange with `invalid_request`.
|
||||
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
|
||||
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
|
||||
POST-body client secrets are not accepted — authenticate the confidential
|
||||
client with `Authorization: Basic base64(client_id:client_secret)`.
|
||||
- **Provider resolution is by `client_id` query param** on the global
|
||||
`/application/o/authorize/` route; per-slug authorize paths do not exist in
|
||||
this build. Token/userinfo live at `/application/o/token/` and
|
||||
`/application/o/userinfo/?api=1`.
|
||||
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
|
||||
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
|
||||
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
|
||||
becomes the *expression source* instead of the context; every call raises a
|
||||
swallowed `PropertyMappingExpressionException` and the claim is skipped.
|
||||
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
|
||||
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
|
||||
so `filter(provider=...)` always yields an empty queryset and tokens asked
|
||||
to carry scopes get zero. Net effect: userinfo/introspection answers with
|
||||
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
|
||||
build until the evaluator is patched or upgraded. Clients consuming this
|
||||
provider must key identity off `sub`.
|
||||
- Scope mappings for this provider exist — `grafana-email-scope` and
|
||||
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
|
||||
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
|
||||
the evaluator bug is fixed; no config change is needed then.
|
||||
The live proxy-provider inventory (the per-vhost `external_host` → app mapping)
|
||||
the two provisioning rules the outpost enforces, and this fork's OAuth2
|
||||
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
|
||||
|
||||
## LDAP outpost (Jellyfin auth, live 2026-09-01)
|
||||
|
||||
The second outpost — `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
|
||||
→ `authentik-outpost-ldap.service`, one of the five units in this stack —
|
||||
serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`,
|
||||
serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`
|
||||
`Restart=always`, container up, 2026-10-18). The image is pinned by its **amd64
|
||||
single-platform digest**
|
||||
`ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9`
|
||||
@@ -260,51 +159,16 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
|
||||
.digest'` and pin that digest.
|
||||
|
||||
- Ports **389** (ldap) and **636** (ldaps) publish on the LAN
|
||||
(`PublishPort=389:3389`, `636:6636`,
|
||||
(`PublishPort=389:3389`, `636:6636`
|
||||
`dankclaw/authentik/quadlet/authentik-outpost-ldap.container:30-31`; `ss -ltnp`
|
||||
shows `0.0.0.0:389` and `0.0.0.0:636`, 2026-10-18) and Dankserver can reach
|
||||
them — a bare TCP connect from Dankserver to `192.168.0.143:389` succeeds
|
||||
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
|
||||
from the host `dankclaw/.env` (git-ignored).
|
||||
|
||||
### Authentik-side objects (provisioned via `ak shell` ORM)
|
||||
|
||||
Read from the live database (read-only `SELECT`s against the
|
||||
`authentik-postgres` container, 2026-10-18):
|
||||
|
||||
| Object | Value |
|
||||
|---|---|
|
||||
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
|
||||
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
|
||||
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
|
||||
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
|
||||
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
|
||||
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
|
||||
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
|
||||
|
||||
Hard-won rules (the first is now contested by the live database):
|
||||
|
||||
- **Token intent: contested, do not trust either value blindly.** This page
|
||||
long asserted that a bind token with `intent=app_password` is rejected by the
|
||||
flow's password stage with `Invalid password` and only `intent=verification`
|
||||
passes. The live token for `svc-jellyfin-ldap` is `app_password`
|
||||
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
|
||||
current Jellyfin bind path is not that token. Nothing observable settles it
|
||||
today: the LDAP outpost's retained log holds only its 5-minute
|
||||
`initialized direct binder` / `Update providers` config cycles and no bind
|
||||
activity *(unverified 2026-10-18: no bind was performed — reproducing it
|
||||
requires the bind secret)*. Re-verify with a real bind before changing the
|
||||
token's intent either way.
|
||||
- **Search needs `search_full_directory`.** Without the RBAC grant the
|
||||
outpost returns only the bind user itself — a `(memberOf=…)` filter then
|
||||
matches zero entries and the plugin logs `Found no users matching X`.
|
||||
The permission is object-level on the provider; grant it via an RBAC role
|
||||
and retest.
|
||||
- `ak shell` import path for the provider model in 2026.8:
|
||||
`from authentik.providers.ldap.models import LDAPProvider`.
|
||||
*(unverified 2026-10-18: importing it outside a configured Django shell fails
|
||||
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
|
||||
auth service for this check)*
|
||||
The objects behind it — LDAP provider, bind DN, service account and bind token,
|
||||
RBAC grant, groups, application slug — and the hard-won bind rules are in
|
||||
[[Authentik Outpost Reference]].
|
||||
|
||||
## Configuration (secrets)
|
||||
|
||||
@@ -332,7 +196,7 @@ every outpost session (see the ORM playbook).
|
||||
core quadlets, starts postgres health-gated (`wait_container_healthy
|
||||
authentik-postgres 120`), then the server (300 s health wait) and the worker.
|
||||
The outposts are separate handlers: `--service authentik-outpost` and
|
||||
`--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`,
|
||||
`--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`
|
||||
dispatch at `:1579-1588`).
|
||||
2. Open `https://auth.franksdank.cloud` — the initial setup wizard runs. It
|
||||
creates the superuser (email + password) and generates the DB schema.
|
||||
@@ -340,12 +204,13 @@ every outpost session (see the ORM playbook).
|
||||
Applications** → add an application + a **Proxy** provider whose
|
||||
`external_host` is exactly the browser-facing origin
|
||||
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
|
||||
flows set (see the provisioning pitfall above). Attach the provider to the
|
||||
flows set (see the provisioning rules in
|
||||
[[Authentik Outpost Reference]]). Attach the provider to the
|
||||
embedded outpost.
|
||||
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
|
||||
the app with the correct redirect URI, and paste the `client_id` /
|
||||
`client_secret` into the app's SSO settings. The relying parties that exist
|
||||
today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14);
|
||||
today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14)
|
||||
**Gitea has no provider row at all** (live provider table, 2026-10-18) — it
|
||||
authenticates itself and stays off the chain by design
|
||||
(`dankclaw/Caddyfile:110-120`).
|
||||
@@ -373,53 +238,12 @@ every outpost session (see the ORM playbook).
|
||||
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
|
||||
Authentik migrates the schema on startup.
|
||||
|
||||
## Self-serve user-settings flow (password change)
|
||||
|
||||
Users change their own account details through the `default-user-settings-flow`
|
||||
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
|
||||
target of the homepage footer's "account" link. It is a `stage_configuration`
|
||||
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
|
||||
user.
|
||||
|
||||
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
|
||||
insertion order):
|
||||
|
||||
| order | stage | type | what it does |
|
||||
|---|---|---|---|
|
||||
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
|
||||
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
|
||||
|
||||
**Password change is a prompt field, not a stage.** The `UserWriteStage`
|
||||
writes whatever the preceding `PromptStage` collected: for the key `password`
|
||||
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
|
||||
is a `password` field on the `default-user-settings` prompt
|
||||
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
|
||||
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
|
||||
`validate_password` runs `authenticate()` against the backends to check the
|
||||
*current* password; it never sets a new one. (A `PasswordStage` was added on
|
||||
2026-09-25 and removed the same day for exactly this reason.)
|
||||
|
||||
Two ORM gotchas: `Flow.stages.add()` fails on this through model
|
||||
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
|
||||
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
|
||||
and a prompt field is a `Prompt` row linked through `PromptStageField`
|
||||
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
|
||||
|
||||
**A blank password must not set an empty password.** `set_password("")` would
|
||||
make the account log in with no password, so a validation policy
|
||||
(`default-user-settings-no-empty-password`, bound to the prompt via
|
||||
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
|
||||
is `""` and allows it when set. Verified with the `PolicyEvaluator`
|
||||
(empty → deny, set → allow, absent → allow) and end-to-end by driving
|
||||
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
|
||||
invalid).
|
||||
|
||||
## Outpost claim cache (`proxysession`) — 403 after a group add
|
||||
|
||||
The embedded outpost does not re-evaluate group membership per request. Each
|
||||
completed authorize writes one row to `authentik_providers_proxy_proxysession`
|
||||
whose `session_data` JSON holds the claims (`groups` included) that get served
|
||||
for that browser session until the row expires (observed TTLs 1–24 h per
|
||||
for that browser session until the row expires (observed TTLs 1–24 h
|
||||
row). A session authorized **before** a user was added to a group keeps
|
||||
serving the old group list — Caddy-side RBAC header gates then 403 despite
|
||||
correct membership, and no authentik event is generated (the 403 happens at
|
||||
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
|
||||
(the inventory table carries both bare origins and trailing slashes;
|
||||
routing matches on hostname, verified 2026-09-02).
|
||||
`internal_host = http://caddy:8017`, all three flows set, and an
|
||||
Application attached — the two provisioning rules above.
|
||||
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
|
||||
2. **Route flip in ONE commit/deploy**: the vhost becomes
|
||||
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
|
||||
inside the `http://:8017` listener add `@<svc> host <vhost>` +
|
||||
@@ -485,7 +309,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
|
||||
000 (connection refused) — the post-auth listener lives in the caddy
|
||||
container's netns; probe inside it or grep the config. Non-browser clients
|
||||
cannot solve the JS challenge — before flipping, audit service-to-service
|
||||
traffic: ingestion/push paths must target internal addresses (`loki:3100`),
|
||||
traffic: ingestion/push paths must target internal addresses (`loki:3100`)
|
||||
never a public hostname of a guarded service. Exempt-by-construction
|
||||
categories stay unguarded: the IdP itself (`auth.`), git smart-HTTP
|
||||
endpoints, services with their own complete auth — document the exemption
|
||||
@@ -512,31 +336,18 @@ EOF
|
||||
`sed -n '/objects imported/,$p'` to drop the noise *(unverified 2026-10-18:
|
||||
the exact banner text and the 229-object count — no `ak shell` was driven
|
||||
against the live auth service for this audit)*.
|
||||
- Import paths (each wrong guess costs a failed exec): `User`, `Group`,
|
||||
- Import paths (each wrong guess costs a failed exec): `User`, `Group`
|
||||
`Application`, `ApplicationEntitlement`, `PropertyMapping` →
|
||||
`authentik.core.models`; `ProxyProvider` →
|
||||
`authentik.providers.proxy.models` (NOT `authentik.outposts.models`);
|
||||
`LDAPProvider` → `authentik.providers.ldap.models`;
|
||||
`authentik.providers.proxy.models` (NOT `authentik.outposts.models`)
|
||||
`LDAPProvider` → `authentik.providers.ldap.models`
|
||||
`authentik.core.models.application` does NOT exist as an import path; there
|
||||
is no `authentik.policies.group` module in this build.
|
||||
*(unverified 2026-10-18: these module paths were not re-executed — importing
|
||||
- (unverified 2026-10-18: these module paths were not re-executed — importing
|
||||
outside a configured Django shell fails with `ImproperlyConfigured`)*
|
||||
- For live DB truth prefer the postgres container, and it works without any
|
||||
credential — the local socket is trusted, so a read-only audit is just
|
||||
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
|
||||
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
|
||||
before guessing: 2026.x renamed both tables and columns, and the models are
|
||||
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
|
||||
three `*_flow_id` columns; the proxy table is
|
||||
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
|
||||
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
|
||||
table in 2026.8), the LDAP table is
|
||||
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
|
||||
`authentik_core_application` attaches to a provider through `provider_id`
|
||||
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
|
||||
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
|
||||
is the same story from the ORM side: `type` lives in the subclass tables.
|
||||
Never select secret columns (`client_secret`, `key`, `password`) into a log.
|
||||
- The live database's table and column names (the 2026.x renames, the MTI parent
|
||||
keys, the read-only `psql` audit recipe) are reference lookup:
|
||||
[[Authentik Outpost Reference]].
|
||||
|
||||
## User provisioning playbook (user-level account)
|
||||
|
||||
@@ -565,7 +376,7 @@ downstream —
|
||||
`category: user` panels from `services.json`.
|
||||
- `openwebui_users` / `openwebui_admins` → ApplicationEntitlements
|
||||
**"Open WebUI Users"** and **"Open WebUI Admins"**, both rows attached to the
|
||||
`Open WebUI` application (`authentik_core_applicationentitlement`,
|
||||
`Open WebUI` application (`authentik_core_applicationentitlement`
|
||||
2026-10-18), consumed by the "Open WebUI roles" property mapping → `roles`
|
||||
claim *(unverified 2026-10-18: the group→entitlement binding and the mapping
|
||||
expression were not read; the mapping itself is not visible under that name
|
||||
@@ -581,7 +392,7 @@ downstream —
|
||||
artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21, mitch 22, eric 23 —
|
||||
and every one carries the `<user>@franksdank.cloud` address the template
|
||||
prescribes. Creation dates (2026-10-07 → 2026-10-15) were not re-checked
|
||||
*(unverified 2026-10-18)*. Template: `username` +
|
||||
- (unverified 2026-10-18)*. Template: `username` +
|
||||
`<user>@franksdank.cloud` + display name + `is_active=True` + the password
|
||||
set via `set_password`, membership in `Non-Admin Users` + `jellyfin_users` +
|
||||
`openwebui_users`. `alex` was recreated from scratch on 2026-10-08 into this
|
||||
@@ -589,7 +400,7 @@ downstream —
|
||||
the row carried no MFA device, token, entitlement, or proxy session, and
|
||||
events survive a user delete because `Event.user` is a denormalized JSON
|
||||
field, not a FK).
|
||||
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`,
|
||||
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`
|
||||
`openwebui_admins`, `authentik Admins`, `authentik Read-only`.
|
||||
- Other user-category vhosts (SearXNG, Gitea, Calibre-Web, RomM, Copyparty,
|
||||
Plex, Seerr, Marreta) need no per-user group: any authenticated user
|
||||
@@ -618,8 +429,8 @@ More field gotchas (each cost a failed exec): `ApplicationEntitlement.applicatio
|
||||
can be null (guard with `getattr`); `LDAPProvider` has
|
||||
`search_mode`/`bind_mode`/`base_dn` and lacks `search_base`/`of_groups`/
|
||||
`organization` — the backing table
|
||||
(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`,
|
||||
`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`,
|
||||
(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`
|
||||
`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`
|
||||
`uid_start_number`, `search_mode`, `bind_mode`, `mfa_support`
|
||||
(`information_schema.columns`, 2026-10-18); `OAuth2Provider` has no `.scopes`
|
||||
(scopes are linked through `ProviderPropertyMapping`, which is why
|
||||
@@ -673,5 +484,5 @@ podman run --rm --network container:<container> --entrypoint /bin/busybox \
|
||||
- [Jellyfin](../Dankserver/Jellyfin.md) — consumes the LDAP outpost via its LDAP-Auth plugin
|
||||
- [Network Architecture](../Architecture/Network%20Architecture.md) — the flat naming scheme
|
||||
- Research: `.docaudit/research/auth.md` — the full provider comparison
|
||||
*(the file is gone from the repo and from the host checkout as of
|
||||
- (the file is gone from the repo and from the host checkout as of
|
||||
2026-10-18)*
|
||||
|
||||
@@ -47,7 +47,7 @@ torrent.
|
||||
WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=25s
|
||||
|
||||
The private key is hashed into the VM env file and never enters the repo.
|
||||
NAT-PMP inbound forwarding is available on Proton (gluetun supports it via
|
||||
NAT-PMP inbound forwarding is available on Proton (gluetun supports it
|
||||
`FIREWALL_INPUT_PORTS` + `NATPMP=true`) — deliberately deferred: the e2e
|
||||
gate passes outbound-only, and swarm peers needing an inbound port is a
|
||||
seeding-era concern, not a grab-only one.
|
||||
@@ -86,7 +86,7 @@ are unaffected.
|
||||
got `Unauthenticated`); the live conf was patched and
|
||||
`scripts/deploy-remote.sh` fixed to write the `_PBKDF2` key. Gotcha
|
||||
(live, 2026-09-01): after qB 5.x ran, its conf rewrite left a hash line
|
||||
4.6.7 rejects (`invalid credentials` for every login, including the arrs);
|
||||
4.6.7 rejects (`invalid credentials` for every login, including the arrs)
|
||||
rewriting the same `@ByteArray` line fresh (same password, 100k
|
||||
iterations) restored login. Any future version flip: regenerate the line.
|
||||
- `HostHeaderValidation=false` + `ServerDomains=*`: the socat forwarder
|
||||
@@ -98,7 +98,7 @@ are unaffected.
|
||||
path (no auto-TMM), so the default itself points at `Finished Torrents`:
|
||||
every grab lands there regardless of client behavior; human review moves
|
||||
files onward; nothing writes to the entropy volume.
|
||||
- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`,
|
||||
- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`
|
||||
`tv-sonarr` (all → `Finished Torrents`).
|
||||
- **Download clients** (gateway `10.88.0.1:8080`, user `admin`): all five test
|
||||
green against live qB — sonarr (`tv-sonarr`), radarr, lidarr, readarr
|
||||
@@ -109,12 +109,12 @@ are unaffected.
|
||||
as "Authentication failure" — that kills runtime auth too, not just API
|
||||
creates; the fix was pinning qB itself (below), after which readarr tests
|
||||
200 like the rest.
|
||||
- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`,
|
||||
- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`
|
||||
not `latest`): `latest` resolved to 5.x, whose `204` login breaks
|
||||
Readarr's frozen client; 4.6.7 answers the legacy `200` + `"Ok."` that
|
||||
every client accepts. Pinned in
|
||||
`frankbook-pro/qbittorrent/quadlet/qbittorrent.container`.
|
||||
- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`,
|
||||
- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`
|
||||
gateway ports). Readarr deliberately omitted: prowlarr's indexer-push
|
||||
test to readarr's `/api/v1/indexer/test` 500s (EOL schema drift).
|
||||
Legacy `Prowlarr-*` indexer rows carried in the migrated dankserver DBs
|
||||
|
||||
@@ -46,7 +46,7 @@ The Prometheus datasource is declared in git, not clicked into `grafana.db`:
|
||||
address Caddy proxies to, never the loopback-published port — and carries
|
||||
`timeInterval: 15s` so Grafana's rate windows cannot be shorter than the
|
||||
scrape interval (`prometheus.yml:5`). The `llm-fleet` dashboard is provisioned
|
||||
alongside it by `dashboards/provider.yml` (provider `fleet`,
|
||||
alongside it by `dashboards/provider.yml` (provider `fleet`
|
||||
`updateIntervalSeconds: 30`, `allowUiUpdates: true`).
|
||||
|
||||
A Loki datasource is still **not** provisioned — the tree contains exactly one
|
||||
@@ -69,8 +69,8 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
|
||||
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
|
||||
Basic client auth, form-encoded token body, and the fork's userinfo-claims
|
||||
bug that leaves identity at the `sub` hash only) is in
|
||||
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
||||
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
|
||||
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
||||
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`
|
||||
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
|
||||
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
|
||||
reach Grafana until the evaluator bug is fixed — do not write role mappings
|
||||
@@ -79,7 +79,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
|
||||
the host without the database or an admin password. *(unverified 2026-10-18)*)
|
||||
- **Never point this unit's `EnvironmentFile` at the repo-root `.env`.** That
|
||||
file is a gluetun env file and it carries `HOME=/root`; systemd imports it
|
||||
for the service, podman then looks for its configuration under `/root`,
|
||||
for the service, podman then looks for its configuration under `/root`
|
||||
cannot read it, and the unit dies at start with
|
||||
`stat /root/.config: permission denied`. The unit reads
|
||||
`/home/dank/Containers/grafana/env`, which the deployer seeds with exactly
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Homepage Status Probes
|
||||
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
|
||||
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
|
||||
tags: [homepage, status, probes, e2e, homepage-status]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
@@ -51,7 +51,7 @@ See [[Container Host Reachability]] for the netns rules.
|
||||
published nothing → structurally unprobeable from the host).
|
||||
3. Probe from the **correct vantage**: from a bridge-net container the host
|
||||
is its gateway IP (`/proc/net/route`, e.g. `10.89.1.1` on
|
||||
`dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`),
|
||||
`dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`)
|
||||
which belongs to the default `podman` net (`10.88.0.0/16`). Probe with
|
||||
bash `/dev/tcp` (no `python3`/`ip` in these images); escape loop vars so
|
||||
they expand **inside** `podman exec bash -c`.
|
||||
@@ -83,7 +83,7 @@ python3 -m http.server 8899 --bind 127.0.0.1
|
||||
```
|
||||
|
||||
Then in the browser: load `http://127.0.0.1:8899/index.html`, **wait ~1.5 s**
|
||||
(panels render after the async fetch; checking immediately returns 0 panels),
|
||||
(panels render after the async fetch; checking immediately returns 0 panels)
|
||||
extract via `tab.evaluate` — LLM cards render `.desc-role`/`.desc-spec`/
|
||||
`.desc-inputs`/`.desc-use` from the card's `details` object, other cards a
|
||||
plain `.desc`; `getComputedStyle(el).color` confirms tones. Screenshot for
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
|
||||
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
|
||||
tags:
|
||||
- infrastructure
|
||||
- homepage
|
||||
@@ -22,7 +22,7 @@ The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the stat
|
||||
**Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
|
||||
page (`shared/homepage/docs.html`) — access walkthrough, the running-model
|
||||
guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
|
||||
remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`,
|
||||
remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`
|
||||
`frank-zealot`) moved from admin-only to the **user** page with live model
|
||||
descriptions and a `docs` link to `/docs#models`.
|
||||
|
||||
@@ -59,7 +59,7 @@ without which the picker showed "No results found" for every family account
|
||||
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
|
||||
intelligence)", frank-archon "equivalent to Claude Opus 4.8 / OpenAI-Terra
|
||||
(same era)", frank-zealot "equivalent to last year's frontier level
|
||||
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
|
||||
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic)
|
||||
pinned in `homepage_config.bats`.
|
||||
|
||||
**Protoss rename + Model Information component (2026-10-08):** the model
|
||||
@@ -91,7 +91,7 @@ same renames, api-id mapping, supervision and compaction notes.
|
||||
app card now carries hex-clipped art beside its name (`services.json` `art`
|
||||
field, files under `img/apps/`): 19 brand marks tinted amber from the
|
||||
Simple Icons CDN plus 19 hand-improvised monogram SVGs (amber hex + VT323
|
||||
initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard),
|
||||
initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard)
|
||||
node-exporter, Gluetun, OpenWebUI, Calibre-Web, RomM, Seerr, Loki, Alloy,
|
||||
Alertmanager, Valkey, Prowlarr, Readarr, Lidarr, Tautulli, ComfyUI. Model
|
||||
cards render in the order zealot → dragoon → archon (registry order is the
|
||||
@@ -143,8 +143,8 @@ per-app colors: Copyparty gold, Marreta brick, Grafana orange, Loki
|
||||
indigo, Neo4j network, Alloy violet, Promtail pink, Alertmanager red,
|
||||
Prometheus orange, Node Exporter teal, cAdvisor sky, Guake yellow,
|
||||
Authentik blue, Seer green radar, RomM a purple GameBoy. New per-service
|
||||
fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`),
|
||||
`warn` on OpenWebUI (searches ride the Seerr→SearXNG path),
|
||||
fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`)
|
||||
`warn` on OpenWebUI (searches ride the Seerr→SearXNG path)
|
||||
`model.compaction` (the amber "set this as compaction model" callout
|
||||
frank-zealot carries). frank-dragoon's card portrait is the SC2 unit
|
||||
portrait (fandom wiki `Dragoon_portrait.jpg`, webp→jpeg). The whole
|
||||
@@ -175,7 +175,7 @@ titles are chamfered plates with the diamond glyph inside the plate.
|
||||
Fonts: body is **Share Tech Mono** (legible retro terminal), headings
|
||||
are **Rajdhani 600** (the StarCraft UI face) — VT323 is retired; all
|
||||
three pages load the pair from Google Fonts. Four tiles that had
|
||||
reverted to initials got pictograms: Calibre-Web an open book (orange),
|
||||
reverted to initials got pictograms: Calibre-Web an open book (orange)
|
||||
RomM a purple GameBoy, Copyparty a blue folder with upload arrow,
|
||||
Marreta a crumbling brick wall with a red crack (`img/apps/app-*.svg`).
|
||||
frank-dragoon's card accent is phase-crystal teal (`#5fe0d0`), matching
|
||||
@@ -197,7 +197,7 @@ state — the only animation left is the speaking-voice pulse. The result
|
||||
is flat navy panels on crisp 1px borders: same Protoss direction, half
|
||||
the pixels.
|
||||
|
||||
**Model ids official (2026-10-08):** the card ids `frank-zealot`,
|
||||
**Model ids official (2026-10-08):** the card ids `frank-zealot`
|
||||
`frank-dragoon`, `frank-archon` are now the harness's official model ids
|
||||
in `models.yml`/`config.yml` (see [[OMP Harness]]); the
|
||||
systemd services keep the `frank-archon`/`frank-dragoon`/
|
||||
@@ -205,7 +205,7 @@ systemd services keep the `frank-archon`/`frank-dragoon`/
|
||||
one name per model.
|
||||
|
||||
**Api ids renamed on the wire (2026-10-09):** the promise in
|
||||
`docs.html` is kept — the served model ids are now `frank-zealot`,
|
||||
`docs.html` is kept — the served model ids are now `frank-zealot`
|
||||
`frank-dragoon`, `frank-archon`, so the docs page no longer carries
|
||||
`api id:` annotations, and the curl/opencode/Pi examples send
|
||||
`frank-zealot`. The systemd services keep the old names. `docs.html`
|
||||
@@ -216,7 +216,7 @@ ids, which did not change.
|
||||
clip-paths replaced with `border-radius` (the clipped corners read as
|
||||
gaps in the 1px border); section-title diamonds moved from absolute
|
||||
positioning to a fixed-width inline glyph (aligned, wrap-safe); closed
|
||||
WIP items grey out (dim type, desaturated art, muted border via
|
||||
WIP items grey out (dim type, desaturated art, muted border
|
||||
`:has(.badge.wip-closed)`); the Dragoon tagline is one spliced track
|
||||
(`sfx/dragoon-tagline.ogg` — loudnorm-matched, silence-trimmed, 350 ms
|
||||
beat; the two SC2 source oggs are trashed); privacy notes became
|
||||
|
||||
@@ -188,7 +188,7 @@ sudo systemctl restart systemd-resolved && resolvectl flush-caches
|
||||
```
|
||||
|
||||
The `~` (route-only) prefix is load-bearing: it routes only those domains to
|
||||
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`;
|
||||
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`
|
||||
`resolvectl status` shows `DNS Domain: ~franksdank.cloud ~franksdank.casa`.
|
||||
|
||||
**Layer 2 — browser DoH (the bypass: "curl works, browser times out").** If
|
||||
@@ -236,7 +236,7 @@ Endpoints (all POST, JSON body):
|
||||
`.content.records[]` as some docs claim — parse with
|
||||
`jq -r '.records[] | [.id,.name,.type,.content,(.ttl|tostring)] | @tsv'`.
|
||||
- Delete: `/dns/delete/{domain}/{id}` (body carries apikey+secretapikey).
|
||||
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`,
|
||||
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`
|
||||
`type`, `content`, `ttl`, `prio`).
|
||||
|
||||
Quirks that bite (verified live 2026-08-31):
|
||||
|
||||
@@ -55,7 +55,7 @@ The [[Alloy]] agent on Dankclaw is wired to ship container and journal logs to `
|
||||
The service answers and the API works; there is simply no data in it. Measured on 2026-10-18:
|
||||
|
||||
- `curl -s http://127.0.0.1:3100/loki/api/v1/labels` → `{"status":"success"}` with no `data` key (no label names exist), and a 7-day `query_range` on `{job="systemd-journal"}` returns `"result":[]`.
|
||||
- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`,
|
||||
- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`
|
||||
`chunks`, `compactor`, `tsdb-shipper-active`, `tsdb-shipper-cache`) —
|
||||
`find /opt/loki/data -type f` counted **8 files**, all WAL segments,
|
||||
cluster-seed and delete-request bookkeeping, **no chunk objects**
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Observability Provisioning
|
||||
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
|
||||
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
|
||||
tags: [observability, prometheus, grafana, provisioning, deployment]
|
||||
timestamp: 2026-10-05T00:00:00-07:00
|
||||
---
|
||||
@@ -21,7 +21,7 @@ committed.
|
||||
| `shared/prometheus/rules/llm-fleet.yml` | alert rules + recording rules |
|
||||
| `shared/grafana/provisioning/` | datasources + dashboards (provisioned tree) |
|
||||
|
||||
Targets and datasources reference **container names** (`grafana:3000`,
|
||||
Targets and datasources reference **container names** (`grafana:3000`
|
||||
`loki:3100` — `shared/prometheus/prometheus.yml:52-61`), never
|
||||
loopback-published ports; `alloy` is **not scraped at all** (no job in the
|
||||
config — its `127.0.0.1:12345` UI is only probed by the homepage timer,
|
||||
@@ -47,7 +47,7 @@ from `deploy_observability`:
|
||||
- `copy_grafana_provisioning` → `/opt/grafana/provisioning/` (`:286-300`)
|
||||
|
||||
`scripts/deploy-plan.py` maps `shared/prometheus/**` to `prometheus` (plus
|
||||
`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`),
|
||||
`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`)
|
||||
`shared/loki/**` to `loki` and `shared/alloy/**` to `alloy` (`:257-260`), and
|
||||
`shared/grafana/**` to `grafana` (`:261-264`); changes under
|
||||
`dankclaw/observability/` select the touched unit and its dependency consumer
|
||||
@@ -71,9 +71,9 @@ HTTP probe, not a metrics endpoint.
|
||||
|
||||
**Add an alert/recording rule** — edit `shared/prometheus/rules/llm-fleet.yml`.
|
||||
Current delivered set: **4 alert rules + 2 recording rules** —
|
||||
`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`,
|
||||
`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`
|
||||
`LlmEndpointDown`, plus `job:llm_prefix_cache_reuse:rate5m` and
|
||||
`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`),
|
||||
`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`)
|
||||
confirmed live: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18)
|
||||
reports 4 alerting + 2 recording. Rules reload with the prometheus service.
|
||||
|
||||
@@ -101,7 +101,7 @@ curl -s -u admin:<pw> http://127.0.0.1:3002/api/datasources | python3 -m json.to
|
||||
|
||||
The two Prometheus queries were run on 2026-10-18 and are what every number on
|
||||
this page comes from: 4 alerting + 2 recording rules, and `up` for
|
||||
`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`,
|
||||
`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`
|
||||
`frank-zealot`, `frank-dragoon`, `frank-archon` with the four structural downs
|
||||
listed in [[Prometheus]]. The Grafana query needs the local admin password and
|
||||
was not run here; the filesystem side (`ls /opt/grafana/provisioning/*`) is
|
||||
@@ -114,12 +114,12 @@ table (refused = outage, 404/HTML = config defect).
|
||||
|
||||
## Current delivered state (delivery chain live 2026-09-17, re-verified 2026-10-18)
|
||||
|
||||
Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`,
|
||||
Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`
|
||||
loopback `:9093`) → host-side Signal relay (`scripts/signal-relay.py`, user
|
||||
unit, `192.168.0.143:18093` — the LAN IP, the one bind a container can reach
|
||||
via `host.containers.internal`) → signal-cli, addressed by number
|
||||
`+12062476336` (`scripts/signal-relay.py:36-43`). Every hop re-checked
|
||||
2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`,
|
||||
2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`
|
||||
`curl -s http://192.168.0.143:18093/health` → `ok`, and
|
||||
`journalctl --user -u signal-relay` ends with successful `sent to +12062476336:`
|
||||
lines for `LlmEndpointDown`, `LlmRequestsQueueing`, `LlmSpecDecodeOff` and
|
||||
@@ -148,7 +148,7 @@ state lives in Prometheus' process, not on disk — a restart re-stamps
|
||||
`activeAt` on a long-standing alert, so a "fresh" alert after a reboot is not
|
||||
evidence of a fresh fault. (Alertmanager by contrast **does** persist:
|
||||
`--storage.path=/alertmanager` on a host volume keeps silences and the
|
||||
notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`,
|
||||
notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`
|
||||
`/opt/alertmanager/data`.)
|
||||
|
||||
Remaining known downs are the structural ones above (HTML-scraped probes, Gitea
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "Observability on Dankclaw"
|
||||
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
||||
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
|
||||
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
|
||||
timestamp: 2026-09-11
|
||||
---
|
||||
@@ -51,13 +51,13 @@ Dankserver runs one member of this family, node-exporter, as its own unit
|
||||
## Exposure
|
||||
|
||||
The five HTTP surfaces publish on `127.0.0.1` only — `ss -ltnp` (2026-10-18)
|
||||
shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`,
|
||||
shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`
|
||||
`127.0.0.1:12345` and no wildcard — because each of them is unauthenticated
|
||||
inside, and several carry lifecycle or query APIs. Browser access goes through
|
||||
Caddy: each vhost (`dankclaw/Caddyfile:128-145`) proxies to the guard
|
||||
`anubis:8923`, and the post-auth handlers route by container name on
|
||||
`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`),
|
||||
`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`),
|
||||
`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`)
|
||||
`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`)
|
||||
`prometheus.franksdank.cloud` → `prometheus:9090` (`:379-381`).
|
||||
Alertmanager has no vhost at all. node-exporter is the exception that must live
|
||||
in the host network namespace, and it binds exactly two addresses rather than
|
||||
@@ -70,13 +70,13 @@ and the podman bridge subnet `10.89.0.0/16` before the DROP
|
||||
|
||||
## Deployment
|
||||
|
||||
`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`,
|
||||
`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`
|
||||
which installs the unit, copies that service's config
|
||||
(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`,
|
||||
(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`
|
||||
`copy_alloy_config`, `copy_grafana_provisioning`, plus
|
||||
`install_signal_relay` with the alertmanager deploy), then waits on the loopback
|
||||
health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`,
|
||||
`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`,
|
||||
health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
|
||||
`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`
|
||||
`/node-exporter`: `/metrics`). Startup order follows the data dependencies —
|
||||
`grafana` after `prometheus`, `alloy` after `loki`
|
||||
(`grafana.container:32-33`, `alloy.container:38-39`). See
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
|
||||
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
|
||||
tags:
|
||||
- infrastructure
|
||||
- prometheus
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
- postgres
|
||||
timestamp: 2026-08-28
|
||||
---
|
||||
|
||||
# authentik-postgres
|
||||
|
||||
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
|
||||
OIDC provider. It is the only stateful store in the stack (users, flows,
|
||||
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
|
||||
now live here.
|
||||
|
||||
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
|
||||
- **Image:** `docker.io/library/postgres:16-alpine`
|
||||
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
|
||||
- **Health:** `pg_isready -d authentik -U authentik`
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
|
||||
(`pg_dump`), and failure modes.
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-09-16
|
||||
---
|
||||
|
||||
# authentik-server
|
||||
|
||||
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
|
||||
This is the service the Caddy `forward_auth` vhosts proxy to
|
||||
(`authentik-server:9000`). It also serves the login page at
|
||||
`https://auth.franksdank.cloud`.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
|
||||
- **Port:** `9000` (published; the outpost + admin + login all live here)
|
||||
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
|
||||
- **Health:** `http://localhost:9000/-/health/`
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
|
||||
forward_auth pattern, first-time setup, and failure modes.
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The Authentik worker — background tasks: event processing, provider"
|
||||
tags:
|
||||
- infrastructure
|
||||
- authentik
|
||||
timestamp: 2026-09-16
|
||||
---
|
||||
|
||||
# authentik-worker
|
||||
|
||||
The Authentik worker — background tasks: event processing, provider
|
||||
synchronization, scheduled flows, and certificate renewal. It runs the same
|
||||
image as the server; the `command: worker` selects the role.
|
||||
|
||||
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
|
||||
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
|
||||
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
|
||||
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
|
||||
- **No port** — the worker has no HTTP endpoint.
|
||||
|
||||
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
|
||||
modes.
|
||||
@@ -100,7 +100,7 @@ The DFlash drafter proposes, then ngram-mod runs as an additional implementation
|
||||
`--spec-type` takes a comma list, and the gate is `ctx_dft` in
|
||||
`common/speculative.cpp` (`common_speculative_init_result`): **draftless**
|
||||
types (the ngram family — simple, map-k, map-k4v, mod, cache) are added
|
||||
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
|
||||
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`
|
||||
`draft-dspark`, `draft-eagle3`) all require a draft CONTEXT. Consequences:
|
||||
|
||||
- A chain may hold ONE context-backed drafter plus any number of ngram types.
|
||||
@@ -201,7 +201,7 @@ Speculative decoding is lossless under rejection sampling: the target verifies e
|
||||
On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
|
||||
at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
|
||||
three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
|
||||
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
|
||||
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see ).
|
||||
|
||||
## Draft Models
|
||||
|
||||
@@ -257,7 +257,7 @@ DSpark took `ROCm error: an illegal memory access` inside the shared draft
|
||||
context with two concurrent 16k prompts; DFlash corrupted multi-slot into
|
||||
premature EOS and was retired first. The timeline, the crash class, the
|
||||
"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
|
||||
are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
|
||||
are on ; the shape they were measured on is
|
||||
[[frank-dragoon]]'s mid-August 2026 era.
|
||||
|
||||
## A/B protocol on a live llama.cpp endpoint
|
||||
@@ -303,7 +303,7 @@ Protocol:
|
||||
|
||||
Read the right counters — per request `timings.draft_n` / `draft_n_accepted` /
|
||||
`predicted_per_second`; cumulative `/metrics`
|
||||
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`;
|
||||
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`
|
||||
per-task `draft acceptance = ... (N accepted / M generated)`. `draft_n == 0`
|
||||
while tokens flow = the proposer is NOT engaged, whatever the flags say. A
|
||||
zero-draft config is a useful negative control: it proves the other config's
|
||||
@@ -367,7 +367,7 @@ states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
|
||||
|
||||
**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
|
||||
matched pair; the arm that anchors the bottom of it on this fleet is the
|
||||
depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
|
||||
depth-7 `draft-dflash` row of 's spec table
|
||||
(accepted share, rate and VRAM for every depth are in that table). Cheapest
|
||||
proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
|
||||
greedy continuation agreement on identical prompts, two API calls, no weights
|
||||
@@ -381,10 +381,10 @@ drafting.
|
||||
**Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
|
||||
engines refuse external drafters by design: halogen-flash-server's
|
||||
`DRAFTER_CAP` gates `dflash2` on the loaded checkpoint and an unavailable
|
||||
name is a deliberate 400 (a silent downgrade would make comparisons lie);
|
||||
name is a deliberate 400 (a silent downgrade would make comparisons lie)
|
||||
there is no device routing in its loopback token protocol, so a drafter can
|
||||
never sit on another card. Pure llama.cpp DOES route the draft:
|
||||
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`,
|
||||
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`
|
||||
`-otd`, `-ctkd/-ctvd`, `--spec-draft-n-max` (default 3), `--spec-draft-p-min`.
|
||||
One process sees only the devices its backends were built with — CUDA+Vulkan
|
||||
is the combination that shows a CUDA card AND an AMD iGPU (HIP+CUDA cannot
|
||||
@@ -411,20 +411,16 @@ measurement that would settle it.
|
||||
|
||||
- Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
|
||||
- DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
|
||||
- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
|
||||
-The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by .
|
||||
- `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
|
||||
- Use `rejection_sample_method` default `strict` for lossless output.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Quantization by Hardware Architecture]]
|
||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
||||
- [[Dankclaw System Specs]]
|
||||
- [[Dankserver System Specs]]
|
||||
- [[VLLM Overview]]
|
||||
- [[deployment]]
|
||||
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
|
||||
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
|
||||
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
|
||||
- where the drafter's KV actually lands
|
||||
- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle
|
||||
@@ -30,8 +30,8 @@ taken in.
|
||||
|
||||
- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
|
||||
wins prefill and Vulkan wins decode, and the decode gap widens with context:
|
||||
the 64k A/B is [[qwen3.8-strix-optimizations]] §4, the 262k A/B is
|
||||
[[qwen3.8-spec-matrix]] §High-Context Backend Comparison. The "HIP
|
||||
the 64k A/B is §4, the 262k A/B is
|
||||
§High-Context Backend Comparison. The "HIP
|
||||
out-performs Vulkan for decode" claim did not hold on the 8060S.
|
||||
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
|
||||
does not exist on this machine (checked 2026-08-15). GPU DPM is already
|
||||
@@ -39,7 +39,7 @@ taken in.
|
||||
- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
|
||||
`frank-zealot` runs now is its own Current shape — this row is not it.
|
||||
- **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
|
||||
(ngram-mod) with draft-mtp first — timeline on [[qwen3.8-spec-matrix]]; the
|
||||
(ngram-mod) with draft-mtp first — timeline on ; the
|
||||
spec setting each endpoint runs today is on its own page.
|
||||
- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
|
||||
once on [[Dankclaw System Specs]] (Memory).
|
||||
@@ -54,8 +54,8 @@ taken in.
|
||||
|
||||
That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
|
||||
"dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
|
||||
recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT; the 76.2 GiB row is an
|
||||
era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`,
|
||||
recorded in §Why Vulkan + GTT; the 76.2 GiB row is an
|
||||
era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`
|
||||
the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
|
||||
carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
|
||||
2026-10-18.
|
||||
@@ -125,9 +125,9 @@ All require root and a reboot for the modprobe change.
|
||||
## Sources
|
||||
|
||||
- [[frank-dragoon]] — measured backend/spec numbers
|
||||
- [[deployment]] — deploy and update process
|
||||
- deploy and update process
|
||||
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
|
||||
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
|
||||
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`
|
||||
`common/arg.cpp`, `common/common.h`
|
||||
- NotebookCheck Ryzen AI MAX+ 395 analysis
|
||||
|
||||
@@ -135,5 +135,3 @@ All require root and a reboot for the modprobe change.
|
||||
|
||||
- [[frank-dragoon]]
|
||||
- [[DFlash DSpark Speculative Decoding]]
|
||||
- [[deployment]]
|
||||
- [[index]]
|
||||
|
||||
@@ -32,7 +32,7 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
|
||||
| Device ID | 0x1586 |
|
||||
| Architecture | gfx1151 (Strix Halo) |
|
||||
| GPU-Visible Memory | ~114 GiB GTT aperture (the budget; `mem_info_gtt_total` = 122,402,963,456 B read 2026-10-18; rocm-smi's smaller SVM carve-out is not the budget) |
|
||||
| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ([[ling-3.0-flash-trial]] §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
|
||||
| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ( §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
|
||||
|
||||
### NPU
|
||||
|
||||
@@ -43,10 +43,12 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
|
||||
| Device | /dev/accel/accel0 |
|
||||
| Driver | amdxdna 0.7.0, in-tree — bound since 2026-10-18 (`lspci -k -s c6:00.1` → `Kernel driver in use: amdxdna`; journal 2026-10-18 01:25:14 `Initialized amdxdna_accel_driver 0.7.0 for 0000:c6:00.1 on minor 0`) |
|
||||
| Firmware | 1.1.2.65 (`/usr/lib/firmware/amdnpu/17f0_11/npu_7.sbin` → `npu.sbin.1.1.2.65.zst`, loaded at boot 2026-10-18 01:25:13) |
|
||||
| Runtime | None installed. FastFlowLM 0.9.45 + XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
|
||||
| Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See [[FastFlowLM NPU Inference]] |
|
||||
| Runtime | Distro XRT only: `libxrt2` + `libxrt-npu2` 1:2.21.75 supply `libxrt_coreutil.so.2`, `libxrt_core.so.2` and the NPU plugin `libxrt_driver_xdna.so.2` under `/usr/lib/x86_64-linux-gnu`; **no `/opt/xilinx/xrt`** (census 2026-10-19), no Ryzen AI Python stack. FastFlowLM 0.9.45 + AMD's XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
|
||||
| Status | Not in service — no NPU consumer is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. The only runtime here that ships precompiled programs this device can execute is halogen's — see [[frank-dragoon]] §The NPU lane |
|
||||
|
||||
The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot]], 2026-09-17 → 10-01).
|
||||
This section is device inventory, not a payload record. The name `frank-zealot` served the FLM NPU endpoint until
|
||||
2026-09-11 and has served Bonsai-2 on the RTX 3090 through the llama.cpp PrismML fork since; the checkpoint, the
|
||||
shape and every era are on [[frank-zealot]], and any figure restated here is the next thing to go stale.
|
||||
|
||||
### ROCm
|
||||
|
||||
@@ -80,7 +82,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
|
||||
| Interface | Type | Speed | IP | Notes |
|
||||
|-----------|------|-------|----|-------|
|
||||
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.143 (DHCP) | Carried the RPC path to Dankserver during the 2026-08-17 pipeline-parallelism trial; no RPC listener runs today (`ss -ltn` shows no :50052, 2026-10-18). Default route via 192.168.0.1. |
|
||||
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See [[Pipeline Parallelism/Ethernet Bandwidth Math]] section 0. |
|
||||
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See section 0. |
|
||||
| docker0 | bridge | - | 172.17.0.1 | Usually down. |
|
||||
|
||||
The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
|
||||
@@ -112,5 +114,3 @@ cable between claw and server is required for 10G.
|
||||
|
||||
- [[frank-dragoon]]
|
||||
- [[Shared Memory Architecture Dankclaw]]
|
||||
- [[frank-redo-2026-08-17]]
|
||||
- [[FastFlowLM NPU Inference]]
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: Dankserver System Specs
|
||||
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
|
||||
description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
|
||||
tags:
|
||||
- dankserver
|
||||
- hardware
|
||||
@@ -30,7 +30,7 @@ read 2026-10-18).
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Board | **ASUS ProArt B850-Creator WiFi NEO** (Rev 1.xx) — swapped in 2026-09-20 |
|
||||
| Previous | ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see [[Dual-R9700-Cooperation]]) |
|
||||
| Previous | ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see ) |
|
||||
| GPU lanes | CPU GPP ports bifurcated **x8/x8** at Gen5: `PCIEX16(G5)_1` → `00:01.1`, `PCIEX16(G5)_2` → `00:01.3` (both bridges: `current_link_width=8`, `current_link_speed=32.0 GT/s`, read 2026-10-18); ~31.5 GB/s per card. Each card sits behind an on-board PCIe switch (`01:00.0`/`02:00.0`, `05:00.0`/`06:00.0` — "Navi 10 XL Upstream/Downstream Port of PCI Express Switch"), so the GPU endpoint itself trains **x16** Gen5 while the CPU-facing leg stays x8 |
|
||||
| Atomic routing | `AtomicOpsCap: Routing+` on every bridge in both GPU chains, `AtomicOpsCtl: ReqEn+` on both endpoints — the condition that ruled out TP=2 for the previous board (2026-09-20 census; not re-readable today — `lspci -vv` needs root) *(unverified 2026-10-18)* |
|
||||
| Free | `PCIEX16(G4)` (chipset, x4), M.2_2, M.2_3 |
|
||||
@@ -55,7 +55,7 @@ read 2026-10-18).
|
||||
| Architecture | gfx1201 (RDNA4) |
|
||||
| VRAM | 32,624 MiB each (`rocm-smi` reports 34,208,743,424 B per card) |
|
||||
| Link | PCIe 5.0, endpoint-trained **x16** each behind the on-board switch; the CPU-facing leg is x8 (`00:01.1` / `00:01.3`, both `current_link_width=8` at 32.0 GT/s, read 2026-10-18). No XGMI. Separate IOMMU groups — `03:00.0` → group **15**, `07:00.0` → group **20** (`/sys/bus/pci/devices/.../iommu_group`, read 2026-10-18) |
|
||||
| P2P | **Live and in use.** The running engine prints "I device: the 2 cards reach each other's memory directly (peer to peer)" (`podman logs frank-archon`, 2026-10-15 boot). The 2026-09-20 census had predicted the opposite — ACS `ReqRedir+` on both root ports, so traffic hairpinning at the root complex — and that prediction is superseded; see [[Dual-R9700-Cooperation]] |
|
||||
| P2P | **Live and in use.** The running engine prints "I device: the 2 cards reach each other's memory directly (peer to peer)" (`podman logs frank-archon`, 2026-10-15 boot). The 2026-09-20 census had predicted the opposite — ACS `ReqRedir+` on both root ports, so traffic hairpinning at the root complex — and that prediction is superseded; see |
|
||||
| PCI | 03:00.0 and **07:00.0** (were `03:00.0` / `0b:00.0` on the B650-E) |
|
||||
| iGPU | Raphael (gfx1036) at **11:00.0** — enumerates as ROCm device 2 (rocm-smi GPU[2], 512 MiB). The retired llama.cpp trios hid it with `HIP_VISIBLE_DEVICES=0,1`; the live radiance engine instead enumerates all three AQL devices and ignores it ("3 devices [0:gfx1201 1:gfx1201 2:gfx1036], using 2 of 3, ignoring gfx1036") |
|
||||
|
||||
@@ -104,8 +104,7 @@ RCCL-backed rebuild needs no new host packages.
|
||||
| Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
|
||||
|
||||
The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
|
||||
hijack `libhsa-runtime64` were purged on 2026-09-15 — see
|
||||
[[frank-medium-rocmfpx]].
|
||||
hijack `libhsa-runtime64` were purged on 2026-09-15 —
|
||||
|
||||
### Active Services
|
||||
|
||||
@@ -117,12 +116,12 @@ host-network podman container answering `curl 127.0.0.1:7770/health` →
|
||||
|
||||
The services this page once called active are gone: `frank-medium.service`
|
||||
(llama-server, 32:33 layer split + DFlash2 drafter) is `not-found` — that rung
|
||||
ran 2026-09-16 → 09-20 and is documented in [[frank-medium-llamacpp-dflash2]];
|
||||
ran 2026-09-16 → 09-20 and is documented in
|
||||
`frank-zealot.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU
|
||||
swap. Its archived unit was moved to `~/.AGENTTRASH/20260915-frank-zealot-retire/host-units/`
|
||||
and the vllm-radiance replica trio to `~/.AGENTTRASH/20260916-vllm-radiance-retire/`
|
||||
— neither path exists today (checked 2026-10-18), so the archive location is
|
||||
unverified. See [[deployment]].
|
||||
unverified. See .
|
||||
|
||||
## Hardware Relevance
|
||||
|
||||
@@ -134,7 +133,7 @@ unverified. See [[deployment]].
|
||||
tokens at 32:33, ~6.3/~5.2 GiB headroom) is history. Either way a batch or
|
||||
context change needs a re-fit and not a guess.
|
||||
- 246 GiB system RAM is **not** headroom: the live shape pins a 43.88 GiB host
|
||||
pool per rank (`FRANK_ARCHON_HOST_POOL_MIB=44928`,
|
||||
pool per rank (`FRANK_ARCHON_HOST_POOL_MIB=44928`
|
||||
`scripts/frank-archon.env:184`) plus the 47.68 GiB n-gram/PLE table in RAM,
|
||||
and `free -g` showed **90 GiB available** with the engine loaded (read
|
||||
2026-10-18). The frank-medium claim that "nothing in the serving path is
|
||||
@@ -147,6 +146,6 @@ unverified. See [[deployment]].
|
||||
## See Also
|
||||
|
||||
- [[frank-archon]] — the active serving runbook
|
||||
- [[Dual-R9700-Cooperation]] — why the PCIe path matters, and what the swap changed
|
||||
- [[deployment]] — the deploy flow
|
||||
- why the PCIe path matters, and what the swap changed
|
||||
- the deploy flow
|
||||
- [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory
|
||||
|
||||
@@ -1,327 +0,0 @@
|
||||
---
|
||||
type: research
|
||||
title: Dual R9700 Cooperation
|
||||
description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
|
||||
tags:
|
||||
- frank-medium
|
||||
- rocm
|
||||
- rccl
|
||||
- pcie-atomics
|
||||
- tensor-parallelism
|
||||
- dankserver
|
||||
timestamp: 2026-09-16
|
||||
---
|
||||
|
||||
# Dual R9700 Cooperation
|
||||
|
||||
Dankserver's two R9700s run **one tensor-parallel engine across both cards** —
|
||||
today that is `frank-archon` under radiance 1.0.13, TP=2
|
||||
([[frank-archon]]). When this page was written the endpoint
|
||||
was `frank-medium`: first llama.cpp's layer split
|
||||
([[frank-medium-llamacpp-dflash2]], 2026-09-16 → 09-20), then from 2026-09-20
|
||||
one vLLM-radiance process at `tensor_parallel_size 2`
|
||||
([[frank-medium-radiance-mxfp4]], 2026-09-20 → 09-26). This page records why
|
||||
true tensor parallelism was impossible on the previous board (TUF GAMING B650-E
|
||||
WIFI), and what the 2026-09-20 board swap changed.
|
||||
|
||||
## The short answer
|
||||
|
||||
**The wall is gone.** On the B650-E, card #2 sat behind the chipset, whose
|
||||
bridges could not route PCIe atomic operations. RCCL's kernels carry a hidden
|
||||
hostcall-buffer parameter and ROCr refuses to supply it to a device without
|
||||
atomics, so *no* RCCL collective or P2P operation could be dispatched on that
|
||||
card — see the [measured matrix](#measured-matrix) and the
|
||||
[2026-09-16 table](#the-same-gate-as-this-board-reports-it-2026-09-16) below.
|
||||
|
||||
On 2026-09-20 the board became an **ASUS ProArt B850-Creator WiFi NEO**, which
|
||||
bifurcates the CPU's GPP ports **x8/x8**: the board-swap path this page
|
||||
recommended, at B850 rather than X870E money. Both cards now hang off CPU root
|
||||
ports (`00:01.1`, `00:01.3`), every bridge on both chains reads
|
||||
`AtomicOpsCap: Routing+`, and **both endpoints read `AtomicOpsCtl: ReqEn+`** —
|
||||
the exact precondition this page's own test plan named, and it is now
|
||||
[measured](#measured-verdict-2026-09-20) rather than assumed.
|
||||
|
||||
What that bought:
|
||||
|
||||
- **Both cooperation modes now work, and they were both run.** vLLM TP=2 booted
|
||||
and served (its R4D allreduce needs no RCCL collective at all), and llama.cpp
|
||||
`--split-mode tensor` booted and served against an RCCL-backed build — so a
|
||||
collective HAS been dispatched on this host since the swap.
|
||||
- **TP=2 is the production shape.** It wins on every axis measured, by 2.1x on
|
||||
decode and 3.6x at 4-way concurrency, and it replaced the layer split on the day
|
||||
of measurement. The payload has moved since: the MXFP4 `frank-medium` shape is
|
||||
retired 2026-09-26, and what runs today is the iq4r publisher under radiance
|
||||
1.0.13 ([[frank-archon]]).
|
||||
- **The drafter, not the collective, is what llama.cpp lost.** Its tensor mode
|
||||
could not carry the DFlash2 drafter, which is the honest caveat on the
|
||||
comparison below.
|
||||
- **The bandwidth story moved too.** Both cards ride Gen5 x8 (~31.5 GB/s each):
|
||||
card 0 effectively unchanged (was Gen4 x16), card 1 up ~16x (was Gen4 x1,
|
||||
~1.97 GB/s). The M.2→PCIe riser plan is dead — `PCIEX16_2` holds card #2 now
|
||||
and M.2_2 is free.
|
||||
|
||||
## The gate, in source
|
||||
|
||||
```cpp
|
||||
void Device::checkAtomicSupport() { // rocclr/device/rocm/rocdevice.cpp:181
|
||||
std::vector<amd::Device::LinkAttrType> link_attrs;
|
||||
link_attrs.push_back(std::make_pair(LinkAttribute::kLinkAtomicSupport, 0));
|
||||
if (findLinkInfo(cpu_agent_info_->fine_grain_pool, &link_attrs)) {
|
||||
if (link_attrs[0].second == 1) {
|
||||
info_.pcie_atomics_ = true; // no else branch
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The flag is a **queried hardware property**, consumed at
|
||||
`rocvirtual.cpp:3954` (the hostcall/AQL dispatch path), in `rocblit.cpp` (SDMA
|
||||
copy paths) and exported as `hipDeviceProp.hostNativeAtomicSupported`. There is
|
||||
no environment variable, module parameter, or sysfs toggle around it — verified
|
||||
by grepping ROCclr for `getenv`/`HSA_*`/`ROCR_*`/`AMD_*` in the gate's
|
||||
neighborhood: none. Writing the endpoint's Device Control 2 bit 6 by hand
|
||||
(`setpci -s 0b:00.0 CAP_EXP+28.w=0440`) does nothing because the value is
|
||||
consumed at driver probe time.
|
||||
|
||||
What has to be true upstream of it: every PCIe bridge on the GPU's path to the
|
||||
CPU must advertise `AtomicOpsCap: Routing+`. On this board card #2's path
|
||||
includes `05:00.0` and `06:09.0`, both `Routing-` — a **read-only capability
|
||||
bit**, so no software can enable it.
|
||||
|
||||
### The same gate, as this board reports it (2026-09-16)
|
||||
|
||||
`lspci -vv` on dankserver. Card 0's chain is root port `00:01.1` → `01:00.0` →
|
||||
`02:00.0` → `03:00.0`; card 1's is `00:02.1` → `05:00.0` → `06:00.0` →
|
||||
`06:09.0` → `09:00.0` → `0a:00.0` → `0b:00.0`:
|
||||
|
||||
| Device | Role | `AtomicOpsCap` | `AtomicOpsCtl` | Link |
|
||||
|---|---|---|---|---|
|
||||
| `00:01.1` | CPU root port → PCIEX16_1 | Routing+ | — | x16 |
|
||||
| `03:00.0` | **card 0** | 32bit+ 64bit+ | **ReqEn+** | x16 |
|
||||
| `00:02.1` | CPU root port → chipset | **Routing+** | — | x4 |
|
||||
| `05:00.0`, `06:00.0`, `06:09.0` | B650 bridges | **Routing−** | — | x1 at `06:09.0` |
|
||||
| `09:00.0`, `0a:00.0` | PCIe switch ports | Routing+ | — | x1 (downgraded) |
|
||||
| `0b:00.0` | **card 1** | 32bit+ 64bit+ | **ReqEn−** | x16 |
|
||||
|
||||
Three things fall out of that table, none of them in the source alone:
|
||||
|
||||
- **The CPU side is fine** — `00:02.1` is `Routing+`. The wall is inside the
|
||||
chipset, not the platform, which is why a slot on CPU lanes is the fix.
|
||||
- **The endpoint's own bit differs between the cards, and that is the kernel's
|
||||
doing.** `pci_enable_atomic_ops_to_root()` walks the bridge chain and refuses
|
||||
to set Requester Enable when any bridge cannot route atomics. ROCclr's
|
||||
`pcie_atomics_` and the endpoint's `ReqEn` are two faces of one cause, and it
|
||||
makes `lspci -vv -s <bdf> | grep AtomicOpsCtl` a one-command diagnosis: on
|
||||
this host card 0 reads `ReqEn+` and card 1 `ReqEn−`.
|
||||
- **Card 1's effective link is x1**, not the x16 its own capability block
|
||||
advertises: `06:09.0` trained at x1, so ~2 GB/s is the real ceiling. That is a
|
||||
bandwidth fact rather than an atomics one, and it only matters for
|
||||
collectives — a 27B TP=2 allreduce is ≈5.6 ms/token at x1 against ≈1.4 ms at
|
||||
x4. The layer split's boundary copy does not care.
|
||||
|
||||
## Re-measured 2026-09-20 (ProArt B850-Creator WiFi NEO)
|
||||
|
||||
`lspci -vv` after the board swap. Card 0's chain is `00:01.1` → `01:00.0` →
|
||||
`02:00.0` → `03:00.0`; card 1's is `00:01.3` → `05:00.0` → `06:00.0` →
|
||||
`07:00.0`. **No chipset bridge is in either path** — `00:02.1` now carries only
|
||||
LAN/SATA/WiFi:
|
||||
|
||||
| Device | Role | `AtomicOpsCap` | `AtomicOpsCtl` | Link |
|
||||
|---|---|---|---|---|
|
||||
| `00:01.1` | CPU GPP → PCIEX16_1 | Routing+ | — | Gen5 x8 |
|
||||
| `01:00.0` | card 0 switch upstream | Routing+ | — | Gen5 x8 |
|
||||
| `02:00.0` | card 0 switch downstream | Routing+ | — | Gen5 x16 |
|
||||
| `03:00.0` | **card 0** | 32bit+ 64bit+ | **ReqEn+** | Gen5 x16 (on-card) |
|
||||
| `00:01.3` | CPU GPP → PCIEX16_2 | Routing+ | — | Gen5 x8 |
|
||||
| `05:00.0` | card 1 switch upstream | Routing+ | — | Gen5 x8 |
|
||||
| `06:00.0` | card 1 switch downstream | Routing+ | — | Gen5 x16 |
|
||||
| `07:00.0` | **card 1** | 32bit+ 64bit+ | **ReqEn+** | Gen5 x16 (on-card) |
|
||||
|
||||
- Both endpoints read `CAP_EXP+28.w = 0x1440` — bit 6 set, which is the
|
||||
kernel's `pci_enable_atomic_ops_to_root()` having walked a chain with no
|
||||
`Routing-` bridge. Same one-command diagnosis, opposite answer.
|
||||
- `dmesg` agrees on bandwidth: `252.056 Gb/s available PCIe bandwidth, limited
|
||||
by 32.0 GT/s PCIe x8 link at 0000:00:01.1` (and `.13`) — ~31.5 GB/s per
|
||||
card. DMI confirms the bifurcation: `PCIEX16(G5)_2`, "8x or x8".
|
||||
- `rocm-smi --showtopo` reported `PCIE` between the cards (Navi 48 has no XGMI),
|
||||
they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
|
||||
and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
|
||||
would hairpin at the root complex. **That inference is superseded:** the live
|
||||
engine reports the two cards reaching each other's memory directly, which
|
||||
[[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
|
||||
measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
|
||||
atomics were.
|
||||
- Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
|
||||
with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
|
||||
as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
|
||||
IOMMU posture is unchanged.
|
||||
|
||||
**Still unproven (as of 2026-09-20).** The gate that matters is ROCclr's
|
||||
`checkAtomicSupport()` reading the `kLinkAtomicSupport` link attribute; the ReqEn
|
||||
bit is the same cause seen from the kernel side, but no RCCL kernel had been
|
||||
launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
|
||||
boot that does *not* log `internal AllReduce init failed ... falling back to
|
||||
meta-backend butterfly` — "collectives work here" was a strong expectation, not a
|
||||
measurement. **RCCL never got that measurement**; what settled the operational
|
||||
question instead is production traffic — radiance's TP=2 wire carries cross-rank
|
||||
all-reduces over the direct P2P path and prints its own precision, and the
|
||||
measured `wht6` banner with its exact-step bound is on [[frank-archon]].
|
||||
|
||||
## Measured matrix
|
||||
|
||||
Bare 2-rank `torch.distributed` (`backend=nccl`) on the two R9700s, no vLLM, no
|
||||
model. Rank 0 = card #1 (CPU lanes), rank 1 = card #2 (chipset):
|
||||
|
||||
| Operation | rank 0 (card #1) | rank 1 (card #2) |
|
||||
|---|---|---|
|
||||
| `broadcast` | OK | **FAIL** `DistBackendError` |
|
||||
| `all_reduce` | OK | **FAIL** |
|
||||
| `barrier` | OK | **FAIL** |
|
||||
| `send` / `recv` (P2P) | blocked, then timeout | **FAIL** |
|
||||
| `all_gather` (after failure) | group unusable | — |
|
||||
|
||||
The `AMD_LOG_LEVEL=3` trace names the cause: `Pcie atomics not enabled, hostcall
|
||||
not supported` → `AQL dispatch failed!` → `hipLaunchKernel` returns
|
||||
`hipErrorIllegalState` for `ncclDevKernel_Generic_4(...)`. Nothing in vLLM or
|
||||
RCCL mentions atomics; the runtime log is the only place it appears.
|
||||
|
||||
**`send`/`recv` failing is the decisive result**: it kills pipeline parallelism
|
||||
as an option, because vLLM's PP stage handoff is exactly that primitive.
|
||||
|
||||
## Software escapes: closed, and the last one is now moot
|
||||
|
||||
Every row below was written under the old board. The last row — the only door
|
||||
that was ever open — stopped mattering on 2026-09-20, because the hardware gate
|
||||
it existed to route around no longer applies to this host.
|
||||
|
||||
| Idea | Finding |
|
||||
|---|---|
|
||||
| vLLM's IPC custom allreduce | **Gated off by a table keyed to CUDA sm versions** — `{9.0, 10.0, 10.3, 10.7}`; ROCm reports capability `12.0` for gfx1201, so the gate is `False`. `custom_ar` itself is compiled into the image. |
|
||||
| ...and if that table were patched | Still insufficient: `broadcast`, `barrier`, and the TP logits `all_gather` have no non-RCCL path. |
|
||||
| The radiance fork's own allreduce | Already present and *on by default* (`RADIANCE_USE_R4D_AR`, size gate `RADIANCE_AR_MAX_KB`): a one-shot P2P push allreduce wrapping `CudaCommunicator.all_reduce`, exchanging IPC handles over the **gloo** group. With the gate raised it covers prefill and decode, and `ncclDevKernel` disappears from the profile. But it does not intercept `PyNcclCommunicator.__init__`'s **warmup allreduce**, nor `pynccl_comm.all_gather` — both direct RCCL calls. |
|
||||
| NCCL symmetric memory | Unavailable: the shipped RCCL advertises NCCL 2.30.4 but exposes no `ncclDevComm*` API, so torch's symmetric-memory path cannot run. |
|
||||
| Prefill/decode disaggregation | Needs a KV-transfer connector plus coordination collectives; both out. |
|
||||
| gloo for GPU work | CPU/TCP collectives — orders of magnitude too slow for per-layer TP. |
|
||||
| **Rebuild RCCL so its kernels do not declare a hostcall buffer** | **OPEN — unverified.** The claim is that the hostcall requirement is triggered by device-side `printf`/`assert`, and that `-mprintf-kind=buffered` avoids it. Counter-evidence: RCCL's device source at `src/device/` contains **no live** `printf`, `assert`, or `__assertfail` (checked), so what makes its kernels declare `HiddenHostcallBuffer` is unexplained. The claim traces to search-result summaries, not a primary source. Cheap to settle once a host is available: recompile librccl with `-mprintf-kind=buffered`, dump a kernel's declared arguments (`roc-obj-extract` + `llvm-readelf --notes`) and look for `hidden_hostcall_buffer`; if it is gone, a 2-rank probe should dispatch. **Stop and mask the service first** — see the note below. |
|
||||
|
||||
## Bandwidth: why x4 is enough
|
||||
|
||||
A 27B TP=2 allreduce at batch 4 moves roughly
|
||||
$2 \times \text{batch} \times \text{hidden} \times 2\ \text{bytes} \approx 160\ \text{KB}$
|
||||
per layer per forward pass; across ~62 layers that is **≈10 MB per token-step**,
|
||||
or **~1.4 ms at PCIe 4.0 x4** (~7.9 GB/s), and under 0.4 ms at the Gen5 x8
|
||||
~31.5 GB/s each card now has. TP is compute-bound here, not interconnect-bound.
|
||||
That is why the cheap option (the M.2 riser) and the expensive one (the board
|
||||
swap) were both considered adequate on bandwidth alone — and why the board that
|
||||
landed, at x8/x8, clears the arithmetic with room to spare.
|
||||
|
||||
## Ruled out empirically (do not re-derive)
|
||||
|
||||
- Kernel versions 7.0.0-28 / -30 / -31 — byte-identical failure.
|
||||
- `iommu=pt`, and `amd_iommu=off` is unrelated (that was a Dankclaw prefill knob).
|
||||
- P2P hardware: 45.274 GB/s bidirectional between the cards works fine — plain
|
||||
memory traffic is not the problem; *atomic* routing is.
|
||||
- A from-source librccl rebuild: identical failure at the identical call path.
|
||||
- Every NCCL transport/algorithm knob.
|
||||
- A one-card test proves nothing: NCCL short-circuits at `world_size=1`, which
|
||||
is exactly why the two replicas work at all.
|
||||
- Independent corroboration: StillDeadcode/vllm-radiance Codeberg issue #22 —
|
||||
different host, same image, kernel 7.0.0-30, same `enqueue.cc` failure.
|
||||
|
||||
## Measured verdict (2026-09-20)
|
||||
|
||||
Both cooperation modes were run on this host on 2026-09-20 — the same salted
|
||||
10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
|
||||
masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
|
||||
record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
|
||||
split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
|
||||
page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
|
||||
3.6× at 4-way concurrency over the layer split — and replaced the layer split the
|
||||
day of measurement.**
|
||||
|
||||
Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
|
||||
the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
|
||||
pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
|
||||
it decoded **without** speculation while the other arms decoded with it; and part
|
||||
of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.
|
||||
|
||||
Two numbers elsewhere on this page are therefore not interchangeable:
|
||||
|
||||
- The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
|
||||
with RCCL off, on the old board. It was never a TP decode number, and its
|
||||
paired decode figure is in "The third path" below.
|
||||
- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
|
||||
family on different collective paths; the measured pair lives with
|
||||
[[frank-medium-radiance-mxfp4]]'s arms.
|
||||
|
||||
RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
|
||||
failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
|
||||
why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
|
||||
RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
|
||||
radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
|
||||
RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.
|
||||
|
||||
**Rule for any future multi-GPU collective experiment: stop and mask the
|
||||
production service first**, per the benchmarking policy, or run it on a host that
|
||||
is not serving users. The production service is `frank-archon`, a **root** unit,
|
||||
so on Dankserver that is `sudo systemctl stop frank-archon && sudo systemctl
|
||||
mask frank-archon` (`scripts/frank-archon.service`; never `disable` — that
|
||||
deletes the unit file, and `mask` is what the bench policy prescribes). The
|
||||
2026-09-20 wording below — `systemctl --user stop frank-medium && systemctl
|
||||
--user mask frank-medium`, frank-medium being a rootless quadlet — is retired
|
||||
with that service, which today reports `not-found` (read 2026-10-18).
|
||||
|
||||
This is not theoretical: on 2026-09-16 a bare RCCL P2P probe was run against the live
|
||||
service; the host went hard down ~20 minutes later (no ARP, no ICMP, required a
|
||||
power cycle). The mechanism is not proven, but a hung AQL dispatch wedging the
|
||||
GPU driver is a plausible cause and this host has precedent for going
|
||||
unreachable until a power cycle.
|
||||
|
||||
|
||||
## The third path (production in the layer-split era): llama.cpp's layer split
|
||||
|
||||
Everything above is about *collectives*, and vLLM cannot proceed without them
|
||||
in any of its parallelisms — TP needs a per-layer allreduce, and PP's stage
|
||||
handoff is a p2p `send`/`recv` of the hidden state, the primitive this host
|
||||
also fails. **llama.cpp's `--split-mode layer` needs no collective at all**:
|
||||
layers are distributed across the cards and the boundary activation is a
|
||||
plain per-backend copy, once per forward pass. No RCCL kernel is ever launched
|
||||
on the second card, so the atomics gate never comes up.
|
||||
|
||||
That is what `frank-medium` ran from 2026-09-16 to 09-20
|
||||
([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
|
||||
262144 context. Its measured layer-split rates are that page's; this host's
|
||||
tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
|
||||
decode. **That tensor number is not a prediction for an RCCL-backed build** — it
|
||||
is what tensor split costs when the allreduce degrades to a meta-backend
|
||||
butterfly, and it is the number the board-swap path exists to retire.
|
||||
|
||||
The ranking therefore changed twice. On 2026-09-16: software already gets both
|
||||
cards onto one model, so the hardware fix stopped being a capability question
|
||||
and became a performance one — *tensor* parallelism, ~40 % decode, plus one
|
||||
pooled KV instead of two independent 122k-token pools. On 2026-09-20 the
|
||||
hardware half of that question was answered by the board swap; the llama.cpp
|
||||
measurement half never ran, and `GGML_HIP_RCCL` still defaults `OFF` in
|
||||
`scripts/build-llama-rdna4.sh:41` (re-read 2026-10-18) for exactly the reason
|
||||
this page documents — moot while no llama.cpp shape is deployed on this host.
|
||||
|
||||
## What would change the answer
|
||||
|
||||
- ~~An RCCL build whose kernels stop declaring a hostcall buffer~~ — **moot
|
||||
since 2026-09-20.** That door only mattered while the hardware gate could not
|
||||
be passed; the board swap passes it. The `-mprintf-kind=buffered` hypothesis
|
||||
and its counter-evidence stay in the escape table for the record.
|
||||
- The symmetric-kernel path inside RCCL, if a release ever ships the device API
|
||||
(`ncclDevComm*`) that torch's symmetric-memory path needs. The shipped TheRock
|
||||
nightly (`~/.local/share/rocm-7.14.0a20260624/lib/librccl.so.1`) still does
|
||||
not.
|
||||
- ~~Firmware or an AGESA update that makes the chipset bridges advertise atomic
|
||||
routing~~ — arrived as new silicon instead.
|
||||
- A GPU-side workaround: none found in ROCclr, and the gate is per-device,
|
||||
queried once at probe.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Pipeline Parallelism/index|Pipeline Parallelism]] — two-machine research (the PP variant of this problem)
|
||||
- [[Dankserver System Specs]] — hardware inventory
|
||||
- [[Observability]] — the scrape/dashboard/alert stack around these endpoints
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user