diff --git a/serverAppConfigs-docs/LLM/frank-archon.md b/serverAppConfigs-docs/LLM/frank-archon.md index d5516a3..209c3ef 100644 --- a/serverAppConfigs-docs/LLM/frank-archon.md +++ b/serverAppConfigs-docs/LLM/frank-archon.md @@ -193,6 +193,22 @@ Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-m on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 % decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost 8.9 %. +- **A KV-pool cut is a runtime-headroom lever, never an arena lever.** The 6.1e9 → 3.15e9 B/rank pool cut + (2.75 GiB/rank) bought **zero** extra expert-cache slots. The arena moves with the reserve, one-for-one + (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = **+248 slots**). The shape that pool cut replaced died with + `0 bytes free` on a real 995-token request — that is headroom, not slot count. Read `expert cache space … / N + slots` out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was **14.87 GiB / + 6,496 slots** against `GPU KV cache size: 398,658 tokens`, and it is the POOL that caps resident + conversations, not the arena. +- **MNBT moves the compile RANGE, not the batch width.** The engine sets `num_reqs = min(max_num_batched_tokens, + max_num_reqs)`, so the CUDA-graph compile width is **SEQS + 1** for any `MNBT >= SEQS` — 3 at SEQS 2, 4 at + SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range + ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs; + `--enforce-eager` bypasses the path and loses CUDA graphs with it. +- **The prefix cache proves itself at depth, not in the ledger.** Two concurrent conversations of 196,919 tokens + each answered the marker buried at the head of **their own** prompt with **393,837 tokens resident** — above the + fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13): + decode at 51.5k depth **94.1 cold / 102.3 warm / 117.6 prefix-hit t/s**, draft acceptance **0.7284 / 0.7525**. - **The host pool moves with the slab.** `vram_weights` and `host_pool_mib` serve the per-rank resident set *together*: at the fp8-KV pair (26071/3543) the ledger showed the slab full to 259 KiB of slack, and cutting `vram-weights` by 3.8 GiB to pay for a bf16 pool spills ~3.8 diff --git a/serverAppConfigs-docs/LLM/frank-dragoon.md b/serverAppConfigs-docs/LLM/frank-dragoon.md index a1bb84c..e06fa33 100644 --- a/serverAppConfigs-docs/LLM/frank-dragoon.md +++ b/serverAppConfigs-docs/LLM/frank-dragoon.md @@ -69,7 +69,7 @@ rollback plan — recoverable only until the trash is purged. | Window | Payload | Runtime | Shape | Replaced because | Rollback | |---|---|---|---|---|---| -| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), thinking 1,476 vs 784 mean, 7.4 GiB RAM back | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` | +| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), KL vs the swift15 logits mean 0.155 / median 0.057 with top-1 agreement 85.1 % — two sound models disagreeing on content, not a corrupt forward pass, thinking 1,476 vs 784 mean, 7.4 GiB RAM back | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` | | 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage | | 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | halogen card claimed 31.7 t/s vs 27.6–28.1 measured | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` | | 2026-10-02 → 10-03 | aligned `Q8_0_ROCMFPX_AGENT.gguf` 28,214,425,952 B `f3ef2bdb…` | same fork pin | 1 × 262,144 f16 KV, `draft-mtp` 4 (MTP +123 % decode) | the uncensored file was faster *and* uncensored | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` + `download-frank-dragoon-kingjones.sh` | diff --git a/serverAppConfigs-docs/LLM/frank-zealot.md b/serverAppConfigs-docs/LLM/frank-zealot.md index 8ab48a8..5028636 100644 --- a/serverAppConfigs-docs/LLM/frank-zealot.md +++ b/serverAppConfigs-docs/LLM/frank-zealot.md @@ -118,6 +118,9 @@ re-run the same arms with the box quiet before a figure is carried anywhere else - **Addressable ≠ advertised.** The card is 24,576 MiB but leaves **24,123 MiB**: quote `nvidia-smi`'s `free` field, never `total − used` — the driver's reservation is the difference. - **Exact KV and a 786,432 pool cannot coexist here**: 32,768 B/token ⇒ 24,576 MiB before a single weight loads (the retired "26,112 MiB" note doubled q8_0's scale overhead); the 27B analogue died at **25,088 MiB** with `cudaMalloc failed: out of memory`. Halving the pool funds exact KV; the per-slot window stops at the trained 262,144 (no YaRN here), so a third conversation is a KV-dtype decision, never more context. + The wall's own figures: `cudaMalloc` failed on a **25,088.00 MiB** KV request when the engine's fit projection + asked for **33,198 MiB** against **23,558 MiB** free. It booted f16 at all because the arm env's invented + `KV_CACHE_TYPE` was swallowed by the serve script's `:=f16` default — so a failed window became the measurement. - **Exact KV reads 1.882× the bytes per generated token, not 2×** (32,768 vs 17,408 B/token, 2026-10-18): a decode drop of that order is EXPECTED and must be measured, while prefill should hold — and this service can afford the dtype precisely because its load is the compaction hub's ~170k-token prefill, which is weight-bandwidth bound and never re-reads the cache per generated token. The dtype is paid in decode and concurrency. - Never `--fit` with `parallel > 1` — it maximizes the *total* pool and shrinks each slot: set `CTX_SIZE` explicitly and keep it divisible by `PARALLEL` (the seam enforces it). - **KV rate law.** `full_layers × 4 KV heads × 256 dim` = **8,192 elements per token per cache** on this arch (the engine prints `n_embd_k_gqa = n_embd_v_gqa = 1024`), ×2 B (f16), ×1.0625 (q8_0 — 32 codes plus one 2-byte scale @@ -133,6 +136,11 @@ re-run the same arms with the box quiet before a figure is carried anywhere else `tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because projections drift. + The two boots that priced non-KV: 2 × 262,144 **without** the projector — 20,190 used, KV 12,800 ⇒ non-KV + **7,390 MiB**; 3 × 196,608 **with** it — 22,972 used, KV 14,113 ⇒ non-KV **8,859 MiB**. The difference prices the + vision tower at **~1.4 GiB for a 918 MB file**, because the encoder's activation and compute buffers ride with it. + The first 3-slot deploy (3 × 196,608) booted healthy and left only **1,153 MiB** free — that is how the 2–3 GiB + band got violated in practice, and why the seam prices non-KV at the measured 9 GiB and refuses that shape. - **Ubatch is a prefill knob.** Decode never moves with it and the logical `--batch-size` is neutral; prefill climbs monotonically to **2048** on this CUDA card (+4.6 % at 8.9k depth; 710 → 1,188 t/s), a prompt shorter than the ubatch cannot see the knob at all (the control row), and a verdict never crosses backends — the 2× R9700 split peaks at 512. Each rung costs ~0.6–0.75 GiB of compute buffers; 2048 is the ceiling for a 27B-on-3090 shape. - **A quant rung is paid in decode**: Q5_K_M → Q6_K is +875 MiB of weights and **−10 %** rate on this bandwidth-bound card, and the byte-bandwidth law predicts a rung's rate within 1 %. - **Speculation pays only with room on the card.** The 2026-10-18 arm census (all q8_0 K+V at the 786,432 pool): 3 × 262,144 spec-OFF ubatch 2048 → 17,876 / 6,249 free, 167.2 t/s; spec-ON ubatch 2048 → 22,706 / 1,419, 109.2 t/s; spec-ON ubatch 1024 → 21,400 / 2,725, 213.5 t/s; 2 × 262,144 spec-ON → 213.2; 1 × 262,144 spec-ON → 214.2. That is **+28 % decode whenever the card has room**, and the doors back are one window per conversation, or q8_0 KV with ubatch 1024. The draft's own bill at that shape was 3,072.00 MiB of KV plus ~1,758 MiB of buffers. diff --git a/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md b/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md index 596fd31..a19a0e6 100644 --- a/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md +++ b/serverAppConfigs-docs/Operations/Payload and Disk Reclaim.md @@ -196,6 +196,10 @@ whole host shares. process`); `podman system migrate` panics in 5.7.0. - A same-filesystem trash reclaims no space; root-owned leftovers need `sudo`; never `rm` without explicit permission. +- **A giant image layer is not automatically a model blob.** The 2026-10-17 census found a 121 GB layer + (`16b6663d…`) that read like an undeleted payload; it is the **engine image** — the radiance runtime itself. + Map each layer to what the serving trio actually mounts before deleting anything: that census is both the + disk-reclaim proof and the check that no deleted payload path still backs a live container. ## Related diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 5923d1d..8cd64bc 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -245,7 +245,7 @@ levels x4, reasoning_content, response_format, vision — all PASS. KLD gate vs the BF16 reference: recorded on 1.0.13 (50,235 positions, 197 prompts), scored on the same pin at the w4-legal shape. - Verdict: see runbook [[frank-archon-radiance-swift15-w4dn8]]. + Verdict: see runbook [[frank-archon]]. ## 2026-10-06 - **frank-zealot (Dankclaw): ubatch 2048 — the second rung (llama.cpp, @@ -441,7 +441,7 @@ edge of the 2–3 GiB floor. Deployed `e5893dc`, e2e gate PASSED (2 mini specs flaky-then-passed once the sleeping FrankBook-Pro woke mid-run — WoL took a second burst and ~90 s). Docs: - [[frank-zealot-llama-mimo-q6k-2x200704]] (*Compute-buffer geometry*). + [[frank-zealot]] (*Compute-buffer geometry*). ## 2026-10-03 - **frank-archon: shape F then shape G — MNBT 2048, the CUDA-graph plan term @@ -478,7 +478,7 @@ - Verification seam rewritten to assert the shape by its OUTPUTS (util at the 0.982 law, graph charge below the analytic default and per-graph positive, KV pool >= 400,700 tokens at the measured 7,892 B/token). - Runbook: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. + Runbook: `serverAppConfigs-docs/LLM/frank-archon.md`. ## 2026-10-01 - **frank-archon rolled BACK to Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4; @@ -676,7 +676,7 @@ fork, orcarouter Qwen3.8-Flash-Next-Uncensored-FP8, TP=2) ran a full flag-level A/B battery at the 32k working point; every number is from the server's usage counters (SSE chunks are invalid under spec decode). New - runbook: [[dankserver-qwen38-fp8-expert-offload-tuning]]. + runbook: [[frank-archon]]. - Incumbent stands: MTP depth 4, stock gather 8x16, prefetch depth 1, TUNABLE off — **26.80/25.15/26.81 tok/s** at 166/1087/4064 ctx, acceptance 60.7%; the re-check reproduced it (25.80/24.38/26.63). @@ -1293,7 +1293,7 @@ any repo service. spec-absent-by-measurement, gather-32x64-present, MNBT-1024 tests; the qualified-shape and flags tests flipped to the new env keys. - Overnight redo campaign (prefetch OFF everywhere) tabled in - `/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` + `/serverAppConfigs-docs/LLM/frank-archon.md` with the 2026-10-01 cutover section: gather 32x64 +18.7%, MNBT flat 1024-4096, KV pool law (7,926 fp8 / 14,047 bf16 B/token), arena flat zone, six-slot refusal, bf16 3x262144 boot failure root cause (CUDA OOM at @@ -1408,72 +1408,49 @@ any repo service. ## 2026-10-02 — frank-dragoon cutover: vLLM MiMo -> ROCmFPX Q8_0_ROCMFPX_AGENT + MTP -Frank's order was the highest quality of -`kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` on one full context, "whatever -runtime it requires". The answer is forced by the file: `Q8_0_ROCMFPX_AGENT.gguf` -carries tensor type 103 (`GGML_TYPE_Q8_0_ROCMFPX`, ftype 115), which stock -llama.cpp, gufo and every other loader on the fleet cannot read — only the -kingjones30/ROCmFPX fork can, so the runtime is pinned to commit -`36e9acd40e10a87cd3c3ef8ec734668757dc8520` and built by -`scripts/build-rocmfpx.sh`. Shape: 1 session x 262,144 (native), f16 KV, the -bundled BF16 mmproj, and MTP from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` -(n-max 4, `--spec-draft-device ROCm0`). - -- **MTP is the whole performance story: 17.0-17.1 t/s with, 7.60-7.66 without** - (+123 %), one session, temp 0, production stopped per arm, acceptance 0.462 / - mean length ~1.86, at an 8-15 % prefill tax (325-351 vs 381 t/s). The AGENT - routing variant was chosen over the card's plain Q8_0 because it keeps 187 - output-head tensors at plain Q8_0 instead of 2, which measured MTP acceptance - 0.953 vs 0.911. -- Steady GTT occupancy 63.1 GiB of the ~114 GiB aperture; boot to healthy 16 s. -- Rollback: git tag `frank-dragoon-gufo-q4kxl-20261001`. The retired vLLM-era - fetchers moved to `old/`. Runbook - [[frank-dragoon-rocmfpx-q8agent-mtp]]. +- 2026-10-02 — frank-dragoon: vLLM MiMo → ROCmFPX `Q8_0_ROCMFPX_AGENT` + payload + MTP from the repo's own bundled head, one session at the native + 262,144. See [[frank-dragoon]]. +- The runtime is **forced by the file**: tensor type 103 + (`GGML_TYPE_Q8_0_ROCMFPX`, ftype 115) is unreadable by stock llama.cpp, + gufo and every other loader on the fleet — only the kingjones30/ROCmFPX + fork can read it, so the engine is pinned to commit + `36e9acd40e10a87cd3c3ef8ec734668757dc8520` and built by + `scripts/build-rocmfpx.sh`. The AGENT routing variant was chosen over the + card's plain Q8_0 because it keeps 187 output-head tensors at plain Q8_0 + instead of 2, which measured better MTP acceptance. MTP was the whole + performance story of the shape (roughly +120 % decode, ladder on the + runbook). +- Rollback: git tag `frank-dragoon-gufo-q4kxl-20261001`; the retired + vLLM-era fetchers moved to `old/`. Runbook [[frank-dragoon]]. ## 2026-10-02 — frank-zealot: Q5_K_M, then three contexts + vision, sized by two boots -Three orders in one day on the same endpoint: `bartowski/…Q5_K_M` on llama.cpp -with two contexts ("whatever context size fits the 3090"), then **three -contexts** "as much as can fit while keeping overhead open space to 2-3gb on -the 3090", then **"frank-zealot should have vision"**. - -- **Final shape: `CTX_SIZE=540672 PARALLEL=3` = 3 x 180,224**, f16 K + q8_0 V - under flash attention, `--mmproj` vision, all layers on the 3090. Measured - **21,684 MiB used / 2,441 MiB free** of 24,576 — inside the 2-3 GiB band. -- **The pool came from two boots, not a formula.** 2 x 262,144 with no - projector: 20,190 MiB used, KV 12,800 => non-KV **7,390 MiB**. 3 x 196,608 - with the projector: 22,972 used, KV 14,113 => non-KV **8,859 MiB**. The - difference prices the vision tower at **~1.4 GiB for a 918 MB file** — the - encoder's own buffers ride with it. The first 3-slot deploy (3 x 196,608) - booted healthy but left only 1,153 MiB free, which is how the band got - violated and why the seam now prices non-KV at the measured 9 GiB and +- 2026-10-02 — frank-zealot: Q5_K_M payload, then three contexts + vision + in one day, sized by two boots (`CTX_SIZE=540672 PARALLEL=3`, f16 K + + q8_0 V under flash attention, `--mmproj`). See [[frank-zealot]]. +- **The pool came from two boots, not a formula.** Differencing the boots + prices the non-KV floor — and prices the vision tower at **~1.4 GiB for + a 918 MB file**: the encoder's own buffers ride with it. The first 3-slot + deploy booted healthy but left less than the ordered 2–3 GiB band free, + which is why the seam now prices non-KV at the measured value and REJECTS that shape. -- **Three contexts cost window, and that is the tradeoff Frank was told:** - 180,224 tokens per conversation, not the native 262,144. A longer prompt is - a hard `400 exceed_context_size_error`, so `models.yml` sets - `contextWindow: 180224`. Consequence at the time: frank-archon's then - 183,501-token compaction request did not fit here with summary room — - resolved later the same day when frank-archon's window moved to 200,704 - and its trigger fell to 140,492 (see the shape-D entry below). -- Measured (2026-10-02, this shape): decode **106.8 t/s**, prefill **3,814 - t/s** (~5 % faster decode than 2 x 262,144 — shorter slot, shorter attention - scan), three concurrent requests answered in parallel in 1.7 s with a fourth - queued at 3.0 s, native `tool_calls`, `reasoning_content` populated, - `reasoning_effort: low` accepted, solid-red PNG answered **"Red."** +- Three contexts cost window, and that is the tradeoff Frank was told: + 180,224 tokens per conversation, not the native 262,144; a longer prompt + is a hard `400 exceed_context_size_error`, so `models.yml` follows the + slot, not the card. - **Vision contract gotcha:** `/v1/chat/completions` ignores the legacy - top-level `images` field — that request answered "Brown." for the same red - image, i.e. it generated from text alone. Only the `image_url` content part - reaches the model. `/completion` accepts `images` but this build still - requires `messages`. + top-level `images` field — that request answered "Brown." for the same + red image, i.e. it generated from text alone. Only the `image_url` + content part reaches the model. `/completion` accepts `images` but this + build still requires `messages`. - Deployer gates on the projector too (`LLM_SMALL_MMPROJ`), so a missing - mmproj cannot silently ship a blind endpoint; `MMPROJ_FILE=` (explicitly - empty) in the env file is the documented text-only opt-out and returns ~1.4 - GiB to KV. + mmproj cannot silently ship a blind endpoint; `MMPROJ_FILE=` + (explicitly empty) in the env file is the documented text-only opt-out + and returns the tower's ~1.4 GiB to KV. - Reclaim staged, **not purged, awaiting approval**: - `~/.AGENTTRASH/20261002-retired-llm-payloads/` (~59 GiB of superseded - payloads). Kept in place for rollback: `MiMo-Q6K/`, `MiMo-Q5KM/`, - `Qwen3.8-27B-UD-Q4_K_XL/` and `Qwen3.8-27B-UD-Q8_K_XL/` (the latter holds the - gufo DFlash2 drafter). Runbook [[frank-zealot-llama-mimo-q5km]]. + `~/.AGENTTRASH/20261002-retired-llm-payloads/` of superseded payloads. + Runbook [[frank-zealot]]. ## 2026-10-02 — frank-archon shape D: 3 x 200704, fp8, pool sized for 2 contexts @@ -1483,73 +1460,47 @@ first attempt (`30904ba`, 2 x 200704 + SEQS=2) **crash-looped**; the revert went live as `e460bbb`, and the shape was then re-applied one law at a time on top of the proven-good base, root cause first. -- **ROOT CAUSE: `--max-num-seqs 2` cannot compile this model. It was never OOM, - never MNBT, never the KV dtype.** Four single-variable boot probes against - the then-deployed shape (production stopped for each, candidate booted AS the - service): - - | MAXLEN | SEQS | MNBT | result | - |---|---|---|---| - | 262144 | 2 | 1024 | **Engine core initialization failed** | - | 262144 | 2 | 2048 | **Engine core initialization failed** | - | 262144 | 3 | 1024 | healthy - 792,069 tokens, 3.02x, arena 6,248 | - | 262144 | 3 | 2048 | healthy - the then-deployed incumbent | - - `torch.fx.experimental.symbolic_shapes.ConstraintViolationError: Constraints - violated (L['query_start_loc'].size()[0]) ... specialized it to be a constant - (3)`, raised from AOT `build_guards(strict_error=True)` during - `determine_available_memory -> profile_run -> _dummy_run`, ~7 min in. - `model_runner.py` sizes that input buffer `max_num_reqs + 1` and `_dummy_run` - sets `num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so its width is - **SEQS + 1** for ANY `MNBT >= SEQS` - 3 at SEQS 2, 4 at SEQS 3, exactly the - constant in the message. MNBT only moves the compile RANGE ((1,1024) vs - (1,2048)), which is why the identical error appears at both rungs. - `--enforce-eager` bypasses the path but drops CUDA graphs. The 2026-10-01 - bf16 2 x 262144 failure was this same error class; its message named "KV - dtype and SEQS" together and nobody bisected, so the dtype was blamed - it - was a red herring. -- **THE ARENA LEVER IS `max_model_len`, NOT `--kv-cache-memory-bytes`.** The - planner sizes expert cache as `budget - non-expert - KV reserve - graphs - - overhead`, and the KV reserve is registry-priced at **9190 B/token x - max_model_len**, independent of the pool override: - - | MAXLEN | KV reserve | arena | - |---|---|---| - | 262144 | 2.48 GiB | 14.30 GiB / **6,248 slots** | - | 200704 | 1.91 GiB | 14.87 GiB / **6,496 slots** | - - d(reserve) 0.57 GiB = d(arena) 0.57 GiB = +248 slots. The 6.1e9 -> 3.15e9 - pool cut (2.75 GiB/rank) bought **zero** arena slots. The previous session's - premise - "the pool cut becomes arena, ~7,500 slots" - was wrong; the pool - cut is a runtime-HEADROOM lever (the shape it replaced died with `0 bytes - free` on a real 995-token request). -- **Final shape (option D):** `--max-model-len 200704` x 3 slots, fp8 KV, - `--kv-cache-memory-bytes 3,150,000,000`/rank, MNBT 1024, gather 32x64, no - speculative decode, prefetch off, no `--expert-precise-memory`. The argv - keeps the only slot count that compiles; the POOL caps resident - conversations. Deployed `c769ff2`, healthy in 558 s, e2e 33/33. -- **Measured live, and one prediction corrected.** `expert cache space 14.87 - GiB / 6,496 slots` - the arena prediction landed exactly. But `GPU KV cache - size: 398,658 tokens, Maximum concurrency for 200,704 tokens per request: - 1.99x` (3,146,129,408 B reserved), not the 2.04x the arithmetic predicted: - **the per-token KV price is shape-dependent** - 7,701 B/token at MAXLEN - 262144 (6,100,000,000 B -> 792,069 tokens) vs **7,892** at 200704, ~2.5 % - dearer. Never extrapolate a concurrency figure across geometries. -- **Smoke-verified on the live endpoint:** three concurrent 67-token requests - answered in parallel (~4.1 s each) - the 3 slots are real; native `tool_calls` - (`get_weather {"city":"Seattle"}`); thinking in `reasoning` with clean - `content`; `reasoning_effort` accepts only {xhigh, medium, low} and answers - 400 for anything else (re-measured; the fork's message names the set). +- **ROOT CAUSE: `--max-num-seqs 2` cannot compile this model. It was never + OOM, never MNBT, never the KV dtype.** Four single-variable boot probes + against the then-deployed shape (production stopped for each, candidate + booted AS the service): 262144 x SEQS 2 fails at BOTH MNBT rungs with + `torch.fx.experimental.symbolic_shapes.ConstraintViolationError: + Constraints violated (L['query_start_loc'].size()[0]) ... specialized it + to be a constant (3)`, raised from AOT `build_guards(strict_error=True)` + during `determine_available_memory -> profile_run -> _dummy_run`, ~7 min + in; SEQS 3 is healthy at both rungs. `model_runner.py` sizes that input + buffer `max_num_reqs + 1` and `_dummy_run` sets `num_reqs = + min(max_num_batched_tokens, max_num_reqs)`, so its width is **SEQS + 1** + for ANY `MNBT >= SEQS` — 3 at SEQS 2, exactly the constant in the + message; MNBT only moves the compile RANGE, which is why the identical + error appears at both rungs. `--enforce-eager` bypasses the path but + drops CUDA graphs. The 2026-10-01 bf16 2 x 262144 failure was this same + error class; its message named "KV dtype and SEQS" together and nobody + bisected, so the dtype was blamed — it was a red herring. +- **THE ARENA LEVER IS `max_model_len`, NOT `--kv-cache-memory-bytes`.** + The planner sizes expert cache as `budget - non-expert - KV reserve - + graphs - overhead`, and the KV reserve is registry-priced at **9190 + B/token x max_model_len**, independent of the pool override: d(reserve) + equalled d(arena) exactly, and the multi-GB pool cut bought **zero** + arena slots. The previous session's premise — "the pool cut becomes + arena, ~7,500 slots" — was wrong; the pool cut is a runtime-HEADROOM + lever (the shape it replaced died with `0 bytes free` on a real + 995-token request). +- **Final shape (option D):** the argv keeps the only slot count that + compiles; the POOL caps resident conversations (ledger on + [[frank-archon]]). Measured live, one prediction corrected: the arena + prediction landed exactly, the concurrency one did not — **the per-token + KV price is shape-dependent** (7,701 B/token at MAXLEN 262144 vs **7,892** + at 200704, ~2.5 % dearer). Never extrapolate a concurrency figure across + geometries. Smoke probes confirmed the 3 slots real, native `tool_calls`, + and that `reasoning_effort` accepts only {xhigh, medium, low}, answering + 400 for anything else. - **Harness contract followed the window:** `models.yml` `contextWindow` - 262144 -> **200704** (compaction trigger 140,492 + 36,864 = 177,356 <= - 200,704). Side effect: frank-archon's compaction request now fits inside a - frank-zealot slot again, which it had stopped doing when frank-zealot went to - 3 x 180,224. -- **Comment-only follow-up** `295d578` recorded the measured numbers and was - deployed so the per-service revision marker sits at HEAD; e2e 33/33 again. -- Rollback for the pre-shape-D fp8+MTP4 era stays git tag - `frank-archon-fp8-mtp-20261001`. Runbook - [[dankserver-qwen38-fp8-expert-offload-tuning]]. + 262144 -> **200704**; side effect: frank-archon's compaction request fits + inside a frank-zealot slot again. A comment-only follow-up `295d578` was + deployed so the per-service revision marker sits at HEAD. Rollback for + the pre-shape-D fp8+MTP4 era stays git tag + `frank-archon-fp8-mtp-20261001`. Runbook [[frank-archon]]. ## 2026-10-02 - plex: the readiness gate probed HTTP against a TLS-only service @@ -1559,33 +1510,27 @@ which **cannot** succeed. PMS drops plain HTTP off-loopback - the probe's connection arrives on the container's eth0 - so Plex accepts the TCP connection and then answers with nothing, and curl reports "Empty reply from server" (52). -- Trigger: a bare `./scripts/deploy.sh dankserver` (no `--service`) drains the - whole pending backlog, not just the service you were thinking about. plex's - per-service revision had been behind for long enough that this gate had never - run against the current posture. It burned its full 600 s and failed the run; - every service after plex in the order was left undeployed. +- Trigger: a bare `./scripts/deploy.sh dankserver` (no `--service`) drains + the whole pending backlog, not just the service you were thinking about. + plex's per-service revision had been behind long enough that this gate + had never run against the current posture; it burned its full 600 s and + failed the run, and every service after plex in the order was left + undeployed. - **Plex was healthy the entire time.** `https://127.0.0.1:32400/identity` - returned 200 with the MediaContainer XML in 44 ms while the gate was still - timing out. The in-container `http://127.0.0.1:32400/identity` also answers - 200 - loopback is the one path PMS serves in the clear, which is exactly why + returned 200 with the MediaContainer XML in 44 ms while the gate was + still timing out; the in-container plain-HTTP path also answers 200 — + loopback is the one path PMS serves in the clear, which is exactly why the host-side probe could not see the difference. -- Evidence chain that settled it: `podman exec plex curl http://127.0.0.1:32400` - = 200; `curl http://127.0.0.1:32400` from the host = empty reply; - `/proc/net/tcp` inside the container showed `00000000:7E90` LISTEN plus stuck - ESTABLISHED pairs on `1703590A:7E90` (10.89.3.23, the container's own eth0 - address) that never produced a response. - **The repo already knew this law, in a different file:** - `tests/caddyfile.bats` pins the plex vhosts to a TLS upstream with the comment - "PMS drops plain HTTP off-loopback". The deployer's gate never got the same - treatment. Fixed to `curl -fsSk --max-time 3 https://127.0.0.1:32400/identity` - (`-k`: PMS serves its own self-signed cert on 32400). -- Re-deployed `39f0e0d`: plex recorded deployed, the *arr services recorded - (compose-recorded on this host, no restart attempted), tautulli deployed, - e2e 33/33. Verified after: `plex-https=200`, plex container up, tautulli - healthy, frank-archon untouched (`health=200`, NRestarts=0). -- **Operating lesson: `deploy.sh ` with no `--service` is a fleet-wide - drain.** It restarts every service whose recorded revision is behind HEAD and - whose paths changed since. To touch one service, name it. + `tests/caddyfile.bats` pins the plex vhosts to a TLS upstream with the + comment "PMS drops plain HTTP off-loopback". The deployer's gate never + got the same treatment. Fixed to `curl -fsSk --max-time 3 + https://127.0.0.1:32400/identity` (`-k`: PMS serves its own self-signed + cert on 32400). Re-deployed `39f0e0d`: plex recorded deployed, e2e + 33/33, frank-archon untouched (`health=200`, NRestarts=0). +- **Operating lesson: `deploy.sh ` with no `--service` is a + fleet-wide drain.** It restarts every service whose recorded revision is + behind HEAD and whose paths changed since. To touch one service, name it. ## 2026-10-02 — frank-archon shape E: `--expert-precise-memory` (+1,026 arena slots), a mounted compile cache, and the P2P answer @@ -1674,7 +1619,7 @@ every restart since the cutover. and their logs dumped on the exit path, or a failure looks like a bad flag. The flag was fine both times. - Live: `health=200`, `NRestarts=0`, arena 17.22 GiB / 7,522 slots, KV 398,658 - tokens / 1.99x. Docs: [[dankserver-qwen38-fp8-expert-offload-tuning]]. + tokens / 1.99x. Docs: [[frank-archon]]. - **Deployed `1ac4d48`, e2e 33/33, and the live plan matched the probe exactly:** `'expert_precise_memory': True`, `runtime overhead … 5.25 GiB`, `expert cache space 17.22 GiB / 7522 slots`, `swap region 13.7 GiB / 5986 slots`, `GPU KV @@ -1694,1130 +1639,719 @@ every restart since the cutover. ## 2026-10-11 — frank-archon: unquantized (bf16) KV at 2 × 200,000 -- Ask: two concurrent 200,000-token conversations on exact attention state. - Sized from the live fp8 boot ledger, not arithmetic: the paged groups cost - 7.3125 KiB/token per rank (`kv_attn` 26.00 KiB per 4-token block + - `kv_qsa_bkey` 3.25), so the 3,543 MiB pool bought 386,904 tokens = **1.93 - sessions of the served window**. +- 2026-10-11 — frank-archon: **unquantized (bf16) KV at 2 × 200,000** on + exact attention state, with pool, slab and host pool rebalanced around + the bigger cache. See [[frank-archon]]. - Engine door: `--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so llama.cpp's exact-K / quantized-V pair (what frank-zealot - runs) has **no radiance equivalent**. Measured bf16 rate is **13.8125 - KiB/token = 1.89× fp8, not 2×**: the QSA block key (13 layers, 3.25 KiB per - 4 tokens) does not follow the dtype. Pool formula - `tokens × rate + 110 MiB × seqs (recurrent state) + 448 MiB (prefix - checkpoints)` reproduces both boots within 0.1 %, and is now a test seam. -- Deployed (`0bf26b0` + docs): pool 3543 → **6912** MiB (462,752 tokens; ledger - "backs 2 sequences all at 200000 tokens at once"), weights slab 26071 → - 22208, host pool 40960 → 44928 (the evicted experts land there; pinned pool - slots 7,190 → 10,375), prefix host 4096 → 8192 (the entry grows 58.50 → - 110.50 KiB), TTM floor 140000 → 150000 (pinned ledger ~147 GiB, cap 204,800). - `weights + kv` pinned to the measured ceiling **29,120 MiB/card**; cards - settled 1.85 GiB free — more than the fp8 boot's 1.39, because the total - claim dropped. Half the cost is refunded by the engine: idle KV VRAM is the - expert cache's spillover, and the bigger pool *lends* 6.08 GiB where the fp8 - pool lent 2.69. -- `--max-num-seqs 3 → 2`: the fleet's "seqs 2 cannot boot" law is the retired - tcclaviger vLLM fork's torch.compile `query_start_loc` constraint, not - radiance's — radiance booted seqs 2 in the fp8-native era and the 2026-10-06 - sweep had 2 and 3 tied inside ±3 %. Bonus: the fused all-reduce is declared - over 2 × (1+2) = 6 rows, inside the 25-row bound where the wht6 wire is - bit-exact. -- Verified live (no scratch container, no bench): `/health` 200, served id - `frank-archon`; 2 × 196,919 concurrent conversations each answered the - marker buried at the head of its own prompt (393,837 tokens resident, above - the fp8 pool's 386,904); decode at 51.5k depth 94.1 cold / 102.3 warm / 117.6 - prefix-hit t/s, draft acceptance 0.7284 / 0.7525. -- fp8 control arm through a transient `FRANK_ARCHON_ENV_FILE` drop-in (the - frank-zealot ubatch method, argv + ledger page size verified): prefill - **−29 %** (4,379 → 3,126 cold, 4,864 → 3,333 warm), decode inside the noise - band on two of three passes, TTFT at the full window **+25 %** (the pair: - 110.5 s → 138.1 s), draft acceptance **up 0.661 → 0.740**. The price is - prefill — a 14.8 % smaller resident slab, and prefill is the pass that - touches every expert — the gain is exact state plus acceptance. **Open call - for Frank:** keep bf16, or take the cheap fp8 2 × 200k fit (pool 3,648 MiB, - ≈ free) and leave KV quantized. + runs) has **no radiance equivalent**. +- `--max-num-seqs 3 → 2` booted clean: the fleet's "seqs 2 cannot boot" law + is the retired tcclaviger vLLM fork's torch.compile `query_start_loc` + constraint, not radiance's — radiance booted seqs 2 in its fp8-native era + and the sweep had 2 and 3 tied inside noise. - Harness law re-learned: radiance puts the thinking block in - `reasoning_content`, so a probe that reads `content` on a small budget sees - empty text with `finish_reason: stop` — a healthy server scored 0/2 until the - probe set `chat_template_kwargs.enable_thinking=false`. Tool committed as - `scripts/probe-frank-archon-kv.py`. -- Tests 368/368, e2e 33/33 (one unrelated homepage retry). Docs: - [[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected - (the w4dn8 entry said STAGED, the fp8-native one said ACTIVE). + `reasoning_content`, so a probe that reads `content` on a small budget + sees empty text with `finish_reason: stop` — a healthy server scored 0/2 + until the probe set `chat_template_kwargs.enable_thinking=false`. Tool + committed as `scripts/probe-frank-archon-kv.py`. +- The fp8-vs-bf16 control arms and the open keep-bf16-or-take-cheap-fp8 + call are recorded on [[frank-archon]]; the 2026-10-13 section below (in + file order above) closes that call in favour of bf16. ## 2026-10-15 — frank-archon: publisher iq4r payload swap (MODEL_FILE-only) -- Ask: replace the served model with - `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` — the engine author's - published rad-convert of the OFFICIAL Qwen3.8-Flash-Next (w4nl codebook - experts, ten bf16-protected/layer, int8 trunk, PLE/MTP/vision in-container, - card KL 0.0789 / top-1 92.85 %) — **no other argument changes**, mindful - the operator session runs on the endpoint being swapped. -- Disk: Frank authorized one `rm` — the spent Swift1.5 BF16 source (336 GiB) - and the two-era-old `qwen4exp-fp8-native.rad` (173 GiB) left - `/home/dank/models`; 150 GB → 658 GB free. fp8-native is now rebuild-only - (fetch + build scripts); rollback for THIS swap is the w4dn8 `.rad` - (on disk) by reverting the commit. -- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes - 121,969,901,568 + Hub LFS sha256, `--verify-only` default) as transient - unit `iq4r-fetch`; verified, staged sidecar files kept as provenance. -- Shape carried VERBATIM (25600/5120 pair, 44928 pool, fp8 KV 3 × 200,000, - MTP 2, wht6, MNBT 2048, froggeric v22.5). Tests re-keyed - (`frank_archon_unit.bats` pins the new payload id + sha256, keeps the - w4dn8 rollback provenance), homepage card + trio lore + both indexes - updated; new runbook [[frank-archon-radiance-qwen38-iq4r]]. Commits - `6dcf4a5` (swap) + `f44c398` + `c5ac072` (docs), full bats green. -- Cutover self-severed by design (detached `deploy.sh` survived the session - death when frank-archon restarted); engine healthy in ~100 s. First-boot - ledger: w4nl loads on pin 1.0.13 (no pin bump); kv_attn 26.00 KiB fp8 @ - 607,740 tokens (identical geometry); slab 17,610/17,658 slots per rank - (> w4dn8's 17,548 — fewer bf16-protected experts pack more 4-bit table); - sampler 9 rows; MTP 2/step; prefix tier RECOVERED 900,176 blocks across - the swap (arch+tokenizer unchanged). Vision probe green (reasoning read - the solid-red PNG; the 64-token budget went all to thinking — fleet law). -- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw rebooted - 01:41 and the gluetun boot-race dependency-failed `searxng`, `marreta`, - and `marreta-browser` (jobs consumed while gluetun's start job retried; - `Restart=always` never armed because the services never ran). Cascade - restarted; skill codified (`gluetun-dependent-boot-race-recovery`). - Gate rerunning against the repaired fleet; frank-archon's own health gate - passed every deploy. Open: decode/prefill ladders vs the w4dn8 record, - probe leg D — scheduled probe window. +- 2026-10-15 — frank-archon: publisher iq4r payload swap, `MODEL_FILE`-only + — `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`, the engine author's + published rad-convert of the OFFICIAL Qwen3.8-Flash-Next, **no other + argument changes**, mindful the operator session runs on the endpoint + being swapped. See [[frank-archon]]. +- Disk: Frank authorized one `rm` — the spent BF16 source and the + two-era-old fp8-native `.rad` left `/home/dank/models`; fp8-native is now + rebuild-only (fetch + build scripts); rollback for THIS swap is the + previous `.rad`, on disk, by reverting the commit. +- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes + + Hub LFS sha256, `--verify-only` default) as a transient unit. The cutover + **self-severed by design** — a detached `deploy.sh` survived the session + death when frank-archon restarted; engine healthy in ~100 s, and the + prefix tier RECOVERED across the swap (arch+tokenizer unchanged). +- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw + rebooted 01:41 and the gluetun boot-race dependency-failed `searxng`, + `marreta`, and `marreta-browser` (jobs consumed while gluetun's start job + retried; `Restart=always` never armed because the services never ran). + Cascade restarted; skill codified + (`gluetun-dependent-boot-race-recovery`). frank-archon's own health gate + passed every deploy. ## 2026-10-15 — authentik: two more human accounts (Dankclaw) -- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, no - repo objects.** Each `@franksdank.cloud` in the human-user template - shape (`is_active=True`, display name, password via `set_password` on stdin - only — never argv, so nothing lands in `ps`), groups `Non-Admin Users` + - `jellyfin_users` + `openwebui_users`; no admin group granted. Verified - through the exact PasswordStage code path (`authenticate()` accepted each - password and rejected a wrong one), `app_entitlements(open-webui)` → - `Open WebUI Users` for both (identical to the ridley precedent), and - `is_superuser=False` / `path=users`. Zero `proxysession` rows for either — - groups were set before any login, so no claim purge was due. Downstream: - Jellyfin visibility rides the LDAP outpost cache (expect lag); OpenWebUI - mints their accounts as role `pending` on first login. -- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user template - census extended with pk 22–23. + +- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, + no repo objects**, each `@franksdank.cloud` in the human-user + template shape (password via `set_password` on stdin only — never argv, + so nothing lands in `ps`), verified through the exact PasswordStage code + path; zero `proxysession` rows for either, so no claim purge was due. +- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user census + extended with pk 22–23. ## 2026-10-17 — frank-zealot: Bonsai-2-27B PQ2_0 trial + cutover (2 × 200,704, q8_0 KV) -- Ask: evaluate `prism-ml/Ternary-Bonsai-2-27B-gguf` (PQ2_0) as frank-zealot's - payload, and specifically measure **2 × 200,704 with q8_0 K + q8_0 V** on the - RTX 3090. Fetch script `scripts/download-frank-zealot-bonsai2.sh` (exact - bytes + sha256, `--verify-only`) landed first; payload verified at - `/home/dank/models/Bonsai2-PQ2_0/` (7,206,168,928 B + 629,246,976 B mmproj). +- 2026-10-17 — frank-zealot: **Ternary-Bonsai-2-27B (PQ2_0) cutover**, two + slots at 200,704 over q8_0 K + q8_0 V on the RTX 3090, fetched by + `scripts/download-frank-zealot-bonsai2.sh` (exact bytes + sha256, + `--verify-only`). See [[frank-zealot]]. - Runtime door: the file is `file_type 141` with 402 tensors of custom GGML - type **142** plus `prism.hadamard.*` metadata — stock llama.cpp refuses it, - so the PrismML vendor build 10754 (`2459f68b5`) is forced, not chosen. A - source build with `GGML_CUDA_FA_ALL_QUANTS=ON` was started and then - cancelled as unnecessary: the vendor build already serves q8_0 V under flash - attention at the exact 34,816 B/token (ledger-confirmed). -- Trial-window law cost one boot: the scratch env invented `KV_CACHE_TYPE` - where the serve script reads `CACHE_TYPE_K`/`CACHE_TYPE_V`, so the script's - `:=f16` default silently won and every arm booted f16. That failed window is - now the measured f16 wall: `cudaMalloc` OOM on a 25,088.00 MiB KV request, - engine fit 33,198 vs 23,558 free. -- Four-arm census (production stopped, trap-guarded restore, `bg_12`): - **2 × 200,704 q8_0 = 22,664 used / 1,461 free (DEPLOYED)**; 1 × 262,144 - q8_0 = 18,130 / 5,995 (fits, but serializes compaction); 2 × 262,144 q8_0 - ≈ 25,758 projected (no fit); 2 × 200,704 q4_0 = 16,392 / 7,733 (fallback - rung). Recurrent DeltaNet state bills ~150 MiB per SLOT (context 13,627 − - KV 13,328 for two slots), which pool arithmetic alone misses. -- Behavior probes green: exact-instruction chat, `reasoning_content` present, - `tool_calls` with valid JSON arguments (no fork `KNOWN_ISSUES` loop/500), - 512 px solid-red PNG answered **"Red"**, needle retrieval exact through a - 25,960-token prompt, 166,027-token prefill clean in 240.0 s. Rates: 76.6 t/s - single-stream (9B ran 94.4), 1,258 t/s at 35k, 691.7 t/s at 166k, - **109.4 t/s aggregate** at two concurrent streams. -- New fleet laws recorded (AGENTS.md + runbook): the mmproj's 873.10 MiB - worst-case is NOT inside the engine's fit projection; the first request after - a boot is a warmup artifact (2.6 / 37.7 t/s measured); a 64-token thinking - budget makes a thinking model look blind; the Bonsai-2 template reads - `reasoning_effort` (xhigh/medium/low, raises otherwise) so the harness fold - became load-bearing on this endpoint, and it reads `preserve_thinking`. -- Cutover: `scripts/frank-zealot.env` (payload + `CACHE_TYPE_*`=q8_0 + ledger - lore), `scripts/deploy-remote.sh` (runtime/model/mmproj gates re-keyed, fork - hint), `tests/frank_zealot_unit.bats` (payload pair pinned; KV assertions now - env-derived instead of an f16 literal), harness `models.yml` (extraBody 1.0 - /0.95/20 + `preserve_thinking`, effort-map comment corrected — - `contextWindow` 200704 and `maxTokens` 16384 unchanged), new runbook - [[frank-zealot-llama-prism-bonsai2-pq20]], indexes re-keyed, 9B era retired - in place. Full bats green (371). + type **142** plus `prism.hadamard.*` metadata — stock llama.cpp refuses + it, so the PrismML vendor build is **forced, not chosen**; the vendor + build serves q8_0 V under flash attention, so the source rebuild with + `GGML_CUDA_FA_ALL_QUANTS=ON` was cancelled as unnecessary. +- Trial-window law cost one boot: the scratch env invented + `KV_CACHE_TYPE` where the serve script reads + `CACHE_TYPE_K`/`CACHE_TYPE_V`, so the script's `:=f16` default silently + won and every arm booted f16. A silently-defaulted knob turns a whole + trial window into a different experiment. +- New fleet laws recorded (AGENTS.md + runbook): the mmproj's worst-case + VRAM is NOT inside the engine's fit projection; the first request after a + boot is a warmup artifact; a 64-token thinking budget makes a thinking + model look blind; the Bonsai-2 template reads `reasoning_effort` + (xhigh/medium/low, raises otherwise) so the harness effort fold is + load-bearing on this endpoint, and it reads `preserve_thinking`. + Recurrent DeltaNet state bills ~150 MiB **per slot** — pool arithmetic + alone misses it. - Base alignment: **Frank chose the OFFICIAL censored payload** — publisher - provenance over refusal-freedom. The 9B it replaces was uncensored, so the - homepage card and its warning contract were re-keyed with it: zealot's card - now carries the base-alignment notice, the shared legal warning belongs to - dragoon alone, and the `homepage_config.bats` equality - (`zealot.warn == dragoon.warn`) retired with the payload it described. - The CRACK re-export (byte-identical geometry per header census) stays on - disk as the one-line `MODEL_FILE` fallback if refusals bite. -- **Deployed 19:44** (`--service frank-zealot --service homepage`): production - boot reproduced the census exactly — 22,664 used / 1,461 free, fit projected - 21,439 leaving 2,119 ≥ 1,872, KV 13,328.00 MiB at `n_ctx_seq = 200,704`, - `n_slots = 2`, projector loaded, alias `frank-zealot` on `/v1/models`. Live - probes: exact `pong` + `reasoning_content`, `tool_calls` `{"city":"Seattle"}`, - 512 px PNG → **Red**, 75.8 t/s decode (1 % off the trial arm). e2e 33/33. -- Two live-path stragglers the cutover also had to catch: `serve-frank-zealot.sh` - still DEFAULTED to the 9B paths and f16 KV (a defaulted run would have served - the retired payload — now re-keyed to the shipped trio), and the OpenWebUI - `model` row's `meta.description` is live DB state naming the 9B (re-keyed, - backup `webui.db.bak-20261017a`; no `open-webui` restart forced — the cached - list picks it up on the next one, and forcing it logs every family session - out for one cosmetic string). + provenance over refusal-freedom. The 9B it replaced was uncensored, so + the homepage card and its warning contract were re-keyed: zealot's card + now carries the base-alignment notice, the shared legal warning belongs + to dragoon alone. The CRACK re-export (byte-identical geometry per header + census) stays on disk as the one-line `MODEL_FILE` fallback if refusals + bite. +- Two live-path stragglers the cutover had to catch: + `serve-frank-zealot.sh` still DEFAULTED to the retired payload's paths + (a defaulted run would have served the old model — now re-keyed to the + shipped trio), and the OpenWebUI `model` row's `meta.description` is live + DB state (re-keyed with a DB backup; no `open-webui` restart forced — + forcing it logs every family session out for one cosmetic string). ## 2026-10-17 — dankserver disk purge: Swift w4dn8 rollback + orphan caches (docs re-key) - Ask: "dump the swift rollback and the GLM GGUF — make sure the model caches and agenttrash are empty as well." Preflight first: `lsof` empty - on both payloads, serving trio names only the iq4r path, and a podman + on both payloads, the serving trio names only the live path, and a podman layer census proved no deleted path backs an image container (the 121 GB - `16b6663d…` layer is the ENGINE image, not a model blob). -- Staged to `/home/dank/.AGENTTRASH/20261017-dump-payloads-caches`, then - purged with Frank's approval: the Swift1.5 w4dn8 `.rad` (113.6 GiB, - rollback target), `GLM-4.7-Flash-UD-Q5_K_XL.gguf` (21 GiB, `/opt/vllm` - dir removed — last non-iq4r model on the box), `radiance-state` kvcache - tiers (69 GiB + 61 MiB, derived), `~/.radiance-cache-w4a8-*` (11 GiB - stale-pin compile cache), `~/.cache/frank-primary` (4.7 GiB stale-name - cache). Also emptied every `.AGENTTRASH` on the host (14 dirs: dated - batches, quadlet/config holds, both checkouts, `/etc/caddy`). 543 → 763 - GiB free. -- Recovery: the w4dn8 payload is byte-exact on the Hub as - `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` - (122,071,973,888 B, LFS sha256 `09dd9306…`) — rollback is now a - re-download, not a local file. Corpora (grams 41 G, kld-ref 24 G, - calibration JSONL) deliberately KEPT: re-convert and KLD-gate inputs. -- Post-checks: frank-archon active, `/health` ok; iq4r serving payload - byte-size intact; live kvcache tier (122 MiB) untouched; no process - references a deleted path. + layer is the ENGINE image, not a model blob). Staged, then purged with + Frank's approval: the two retired payloads, derived kvcache tiers, stale + compile caches, and every `.AGENTTRASH` on the host. +- Recovery: the retired w4dn8 rollback payload is byte-exact on the Hub — + rollback is now a **re-download, not a local file**. Corpora (grams, + kld-ref, calibration JSONL) deliberately KEPT: they are the re-convert + and KLD-gate inputs. +- Post-checks: frank-archon active and `/health` ok, serving payload + byte-size intact, the live kvcache tier untouched, no process references + a deleted path. - Docs re-keyed: AGENTS.md fleet row + frank-archon section + both runbook - index rows; iq4r runbook Rollback rewritten (HF re-fetch procedure — the - fetch script's REPO/RAD_NAME/STAGE/DEST/SIZE/SHA256 constants are - hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook - header corrected (was still "LIVE", never updated at retirement) and its - Rollback table rebuilt. + index rows; the live runbook's Rollback rewritten (HF re-fetch procedure — + the fetch script's REPO/RAD_NAME/STAGE/DEST/SIZE/SHA256 constants are + hardcoded, so retargeting at the mirror is a script edit); the retired + runbook's header corrected (was still "LIVE", never updated at + retirement). ## 2026-10-17 — frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool -- Ask: move frank-dragoon to `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated` - with the card's recommended settings, save memory, keep the n-gram table - disk-backed. Census first: the endpoint carries only the `slow` + - `security-reviewer` roles (the interactive workhorse is frank-archon), the - table was ALREADY disk-backed (`mapped 51.2 GB, NOT pinned (Pin::None)`), and - `HALOGEN_FLASH_PIN_TRUNK=0` is rejected by the v2 loader — so the only real - lever was the KV pool: 524,288 → the card's 262,144. +- 2026-10-17 — frank-dragoon: cutover to the native v2 abliterated halogen + payload with the card's memory-saving KV pool; census first: the n-gram + table was ALREADY disk-backed and the only real lever was the KV pool. + See [[frank-dragoon]]. - **Filename trap caught before the fetch:** this repo name carried two - different artifacts. The writer-transplant era's file measured - 66,687,678,464 B; the native 2026-10-11 conversion is 66,687,678,432 B — - 32 bytes apart. Byte-exact gate + LFS sha256 `b5bd925f…` in - `scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the peonist - n-gram/vision/tokenizer trio (the release publishes only the main file; a - hardlink is a regular file, so the no-symlink serve gate passes without a - 47.7 GiB copy). -- **Companion risk adjudicated by the engine, not by argument.** The card does - NOT establish the stock peonist companions as matched to a native conversion - and publishes no source-matched pair, so the A/B ran the checkpoint tools on a - stopped service under one knob set (`scripts/census-frank-dragoon.sh`): - `verify` OK, **NIAH 18/18 vs 17/18**, PPL 3.968 vs 3.877 (+2.3 % — the - finetune's edge is gone), KL vs the swift15 logits mean 0.155 / median 0.057 - with top-1 85.1 % = two sound models disagreeing on content, not a corrupt - forward pass. Behavior battery (`scripts/battery-frank-dragoon.py`) on both - boots: refusals 0/5 both, tools/vision identical, needles hit at both depths, - decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 (+12.6 %), thinking - tokens 1,476 vs 784 (the finetune compressed reasoning ×1.9). -- Boot window run through a transient `FRANK_DRAGOON_ENV_FILE` drop-in — never - a scratch port (one iGPU, one server) — with the stop+mask law and a - trap-guarded restore (`scripts/trial-frank-dragoon.sh`). Masking fails on a - relinked unit file, as documented; stop alone was enough. -- Deployed at 22:20 (`fcd7630`, `--service frank-dragoon`): healthy in ~12 s, - e2e 33/33. Production ledger: 62.9 GiB weights locked (0.8 vision) + 7.2 GiB - KV (ONE 262,144-position pool, 6,656 MiB, two slots) + 5.7 GiB working = - **75.8 GiB in all**, no clamp line, n-gram 47.68 GiB read from disk, prompt - cache ON with 12 entries, MTP depth-2. The trial drop-in was moved to - `~/.AGENTTRASH/20261017-v2abl-trial/` before the deploy so the repo env - governs (`DropInPaths=` empty). Live probes: `27*59` → 1593, tool call - `{"city": "Seattle"}`, 512 px red PNG → Red (320 prompt tokens). -- Files moved: env (payload dir/checkpoint/bytes/sha256/pool + the measured - A/B in the header), serve script + deployer (paths, fetch hints), unit - `Description`, seam (`frank_dragoon_unit.bats`: pool law relaxed from - `pool == slots × ctx` to `ctx <= pool <= slots × ctx`, fetch-script name), - new runbook `frank-dragoon-flash-server-v2-native`, swift15 page re-banned - RETIRED, both indexes + AGENTS.md re-keyed, homepage card, `models.yml` - comment (numbers unchanged: window 217,088 / cap 45,056 — the per-REQUEST - ceiling did not move). OpenWebUI `frank-dragoon` row description re-keyed - (backup `webui.db.bak-20261017b`, no restart forced). -- Two stale claims fixed on the way: the retired-era rows said `200,192` ctx - (that ceiling was raised to 262,144 on 2026-10-07, inside the same era), and - the root index still called the halogen-27B page ACTIVE. -- Cost of the swap, stated plainly: two MAXIMUM-length conversations no longer - share the endpoint (one 262,144-position pool behind both slots), PPL +2.3 %, - thinking ×1.9. Bought: 7.4 GiB of RAM back as page cache for the table long - prompts read, +12.6 % cold prefill, retrieval 18/18, the base checkpoint - itself instead of the finetune. + different artifacts 32 bytes apart (the writer-transplant era's file vs + the native conversion). Byte-exact gate + LFS sha256 in + `scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the + peonist n-gram/vision/tokenizer trio — the release publishes only the + main file, and a hardlink is a regular file, so the no-symlink serve gate + passes without a 47.7 GiB copy. +- **Companion risk adjudicated by the engine, not by argument.** The card + does NOT establish the stock peonist companions as matched to a native + conversion and publishes no source-matched pair, so the A/B ran the + checkpoint tools on a stopped service under one knob set + (`scripts/census-frank-dragoon.sh`) plus a behaviour battery + (`scripts/battery-frank-dragoon.py`): two sound models disagreeing on + content, not a corrupt forward pass. The boot window ran through a + transient `FRANK_DRAGOON_ENV_FILE` drop-in — never a scratch port (one + iGPU, one server) — with the stop+mask law and a trap-guarded restore + (`scripts/trial-frank-dragoon.sh`). Masking fails on a relinked unit + file, as documented; stop alone was enough. +- Cost of the swap, stated plainly: two MAXIMUM-length conversations no + longer share the endpoint, the finetune's perplexity edge is gone, and + its reasoning came out compressed ~1.9×. Bought: RAM back as page cache + for the disk-backed n-gram table, faster cold prefill, clean retrieval, + and the base checkpoint itself instead of the finetune. The seam's pool + law relaxed from `pool == slots × ctx` to `ctx ≤ pool ≤ slots × ctx`. ## 2026-10-17 — frank-dragoon: ceiling and pool cut to 200,704; crawl reset fix -- Ask: lower frank-dragoon's context "to 196k". Settled on **200,704** for BOTH - `HALOGEN_CTX` and `HALOGEN_KV_POOL_POSITIONS` (49 × 4096 — the per-slot rung - frank-zealot runs; the card recommends pool == ctx, and that equality is where - the memory is). Nothing in the fleet needs the bigger window, and the v2 - checkpoint serves anything up to its native 262,144. -- Boot ledger (`7c4dbeb`, `status=deployed`): `reserving the KV pool, 200704 - positions (about 5.5 GiB)`, `62.9 GiB of weights … 5.5 of KV pool, 5.7 of - working memory, 74.1 GiB in all`, prompt cache entries 111 MiB at the new ctx, - no clamp line. My arithmetic predicted 5.4/74 — the ledger is quoted instead. +- 2026-10-17 — frank-dragoon: context ceiling and KV pool cut to 200,704 + (49 × 4096, the per-slot rung frank-zealot runs; nothing in the fleet + needed the bigger window). See [[frank-dragoon]]. - Harness moved WITH the ceiling (400-zone law): `models.yml` - `contextWindow: 155648` = ceiling − 45,056; compaction now fires at 85 % = - 132,301 est ≈ 136k measured, and the 180k compaction target of the 262,144 era - is unreachable because the window itself is smaller. `maxTokens` 45,056 and the - coupled cap/timeout 65536/7200 untouched. Worst-case harness request = - 1.03 × est + min(45056, 151552 − est) ≤ 156.2k tokens vs the 200,704 ceiling. -- Probes on the live endpoint: over the ceiling → `400 … it is 297109 tokens and - the context is 200704, 96405 tokens over` in 0.86 s (preflight rejects before - any prefill, so probing costs the endpoint nothing); harness-shaped admission → - 122,679-token prompt with `max_tokens: 45056` (reservation 167,735) accepted, - 81.1 s wall (≈1,513 t/s cold prefill), `27*59` answered **1593** from inside a - 122k-token haystack, `finish_reason: stop`. -- Seam re-keyed (`frank_dragoon_unit.bats`): the fit-law test now proves the + `contextWindow` = ceiling − `maxTokens`; compaction fires at 85 % of the + window, and the 180k compaction target of the 262,144 era is unreachable + because the window itself is smaller. Seam re-keyed to prove the invariant at the new pair — `ctx ≤ pool ≤ slots × ctx` and - `harness window ≤ ctx − 45056` — so a future raise of one without the other - fails the gate. bats 371/371. -- Deploy verdict: the endpoint dispatched healthy (`status=deployed`, `/health` - 200), but the `[3/3]` Playwright gate failed on `12-crawl` and the deployer - correctly said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was - Plex Web, signed out in the `qa-e2e` profile, running its own sign-in flow, - which broke the spec two different ways: - 1. it redirects the top frame, so Chromium cancelled the crawl's in-loop - `page.goto("/")` with `net::ERR_ABORTED` — and that reset sat outside the - `try`, so one host aborted the whole crawl (once as a 420 s timeout, once at - 5.4 s); fixed with `backHome()`, which re-issues on that one error class. - 2. the same hand-off leaves the browser on `app.plex.tv` (Plex's hosted surface - for a session-less client), so the `hostname == target` law could never - settle — this is what the first full-suite re-run surfaced as `1 flaky`; - fixed with an explicit `HANDOFFS` landing list. - Plex itself was healthy throughout (container up 2 days; 32400 answers HTTPS - 401 / plain-HTTP empty reply, which is what Plex Web authenticates against), - and the guard contract is still asserted for every vhost: challenge cleared, no - authentik `uidField`, status < 500. Both symptoms recorded as gotcha 6. -- Gate after both fixes: **33 passed (1.2 m)**, crawl green on the first attempt - (6.5 s) — the earlier `DEPLOY INCOMPLETE` is cleared, and the deploy marker - stays at `7c4dbeb` because every later commit is docs/tests only, which the - planner treats as a no-op for the service. -- Cost of the cut: two maximum-length conversations never coexist (one - 200,704-position pool behind both slots; two of ≤100k still do). Bought: 1.7 - GiB back as page cache for the 47.68 GiB n-gram table — 9.1 GiB against the - 2026-10-06 shape's 524,288 pool — and a smaller prompt-cache entry. + `harness window ≤ ctx − maxTokens` — so a future raise of one without the + other fails the gate. +- Probes on the live endpoint: over the ceiling → `400 … tokens over` in + 0.86 s — preflight rejects before any prefill, so probing costs the + endpoint nothing. +- Deploy verdict: the endpoint dispatched healthy (`/health` 200), but the + `[3/3]` Playwright gate failed on `12-crawl` and the deployer correctly + said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was Plex + Web, signed out in the `qa-e2e` profile, running its own sign-in flow, + which broke the spec two different ways: (1) it redirects the top frame, + so Chromium cancelled the crawl's in-loop `page.goto("/")` with + `net::ERR_ABORTED` — and that reset sat outside the `try`, so one host + aborted the whole crawl; fixed with `backHome()`, which re-issues on that + one error class. (2) The same hand-off leaves the browser on + `app.plex.tv` (Plex's hosted surface for a session-less client), so the + `hostname == target` law could never settle; fixed with an explicit + `HANDOFFS` landing list. Plex itself was healthy throughout, and the + guard contract is still asserted for every vhost. Both symptoms recorded + as gotcha 6; the gate after both fixes went 33 passed with the crawl + green on the first attempt. ## 2026-10-18 — issue #3 research: AMD PerfOpt retires the reason for `amd_iommu=off` -- Ask: does a recent kernel change remove the IOMMU translation penalty that - justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the XDNA2 NPU come - back? Answered and documented; **no host change** — the fix is not installable - from the Ubuntu 26.04 archive. -- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13, gated by - extended-feature bit 45): merged via the iommu tree 2026-09-24, first release - **Linux 7.4**. It stops GPA→SPA translation for the *privileged iGPU* only and - leaves the IOMMU fully armed for every other device, NPU included. The only - consumer is `amdgpu_device_use_perfopt()`, which requires `AMD_IS_APU` **and the - identity domain** — so PerfOpt rides on top of `iommu=pt`, it does not replace - it. A 2026-09-28 follow-up deletes the `amdgpu.iommu_perfopt` knob and makes it - compulsory. -- Local census (read-only): kernel `7.0.0-38-generic`; **zero** `perfopt` symbols - in its System.map; `linux-image-generic` and `linux-generic-hwe-26.04-edge` both - pin `7.0.0-38.38` → 7.4 needs a mainline PPA or a local build. `CONFIG_IOMMU_ - DEFAULT_DMA_LAZY=y` (STRICT unset). `/dev/accel/accel0` already exists as - `crw-rw---- root:render 261,0` with `dank` in `render`, so the quadlet law is - `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups` — the ComfyUI `/dev/kfd` - pattern, no new udev rule. -- **Three premises of the issue corrected.** (1) The 13–16 % is the *vendor's* A/B - against *their* previous boot line, which our record never identified — we - copied the figure on 2026-09-11 without ever measuring here. Strict flush and - bounce buffering are not the mechanism either (Ubuntu ships lazy DMA, and a - coherent iGPU on unified memory takes no SWIOTLB bounce). What identity mode - actually costs the iGPU is the 1:1 page walk plus **ATS/PRI/PASID/GCR3** being - switched on at attach — the same four the PerfOpt enable path switches back off - (the `skip_caps` label in `attach_device`), and why it arms only "when an - integrated GPU is already in identity mode". (2) The NPU does **not** have to - wait for 7.4: `iommu=pt` enables the IOMMU, which is all `err=-19` asks for. - Making passthrough the AMD default was rejected on the list (Robin Murphy, - 2026-03-26), so `pt` stays the sanctioned route. (3) The 460 tok/s baseline is - stale — the 2026-10-17 native-v2 payload runs **1,513–1,604 t/s cold prefill** — - so the window measures `off` vs `pt` locally and inherits no number. -- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and Phoronix's - **Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % on - Strix Point / Krackan laptops) — the modest arm is the one that describes this - box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU regression disables SVA - outright. -- NPU legality under identity mode: `amdxdna` binds via `iommu_sva_bind_device()` - + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain, independent of the default - domain type (`drivers/accel/amdxdna/aie2_pci.c`); `err=-19` needs only an - *enabled* IOMMU, which `pt` gives. -- Self-correction inside the same session: the first pass — and the first version - of the posted comment — claimed `iommu=pt` alone already avoids the cost. It - does not. Patch v2 1/2 shows an identity attach keeps the page walk and enables - ATS/PRI/PASID/GCR3, so `pt` recovers **most** of the gap and PerfOpt is the last - part of it. The comment was edited to the corrected text. -- Artifacts: new section in - `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` (+6 citations, - front-matter re-keyed to "vendor-claimed, never measured locally"), - `frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a middle - ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, a 4-step - maintenance-window procedure that measures locally instead of inheriting the - vendor number, and the plan posted to Gitea issue #3 (comment 23, then edited). - bats 371/371; commits `7643381` plus this correction. Nothing deployed — no - service was touched. +- Ask: does a recent kernel change remove the IOMMU translation penalty + that justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the + XDNA2 NPU come back? Answered and documented; **no host change** at the + time — the fix is not installable from the Ubuntu 26.04 archive. +- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13): merged + via the iommu tree 2026-09-24, first release **Linux 7.4**. It stops + GPA→SPA translation for the *privileged iGPU* only and leaves the IOMMU + fully armed for every other device, NPU included. Its only consumer + requires `AMD_IS_APU` **and the identity domain** — so PerfOpt rides on + top of `iommu=pt`, it does not replace it. +- **Three premises of the issue corrected.** (1) The 13–16 % is the + *vendor's* A/B against *their* previous boot line — our record copied the + figure on 2026-09-11 without ever measuring here. (2) The NPU does **not** + have to wait for 7.4: `iommu=pt` enables the IOMMU, which is all + `err=-19` asks for (`amdxdna` binds via `iommu_sva_bind_device()` into its + own SVA domain, independent of the default domain type). (3) The 460 + tok/s baseline is stale, so the window must measure `off` vs `pt` locally + and inherit no number. +- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and the + **Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % + on Strix Point / Krackan laptops) — the modest arm is the one that + describes this box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU + regression disables SVA outright. +- Self-correction inside the same session: the first pass claimed + `iommu=pt` alone already avoids the cost. It does not — an identity + attach keeps the page walk and enables ATS/PRI/PASID/GCR3, so `pt` + recovers **most** of the gap and PerfOpt is the last part of it. The + posted comment (Gitea issue #3) was edited to the corrected text. +- Artifacts: a new section in `dankclaw-iommu-translation-off.md` + (front-matter re-keyed to "vendor-claimed, never measured locally"), + `frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a + middle ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, and + a 4-step maintenance-window procedure that measures locally instead of + inheriting the vendor number. Nothing deployed — no service was touched. ## 2026-10-18 — boot line flipped: Dankclaw now boots `iommu=pt` -- Window executed 01:24–01:35 on Frank's go-ahead, with the A/B the research - demanded. Provenance first: `/etc/default/grub` had never been in the repo - while the IOMMU line was the most consequential kernel decision on the box, so - it is now tracked at `dankclaw/host/grub` (fetched byte-exact, 1581 B) and moves - only through `scripts/set-dankclaw-bootline.sh` — diff, install 0644 root:root, - `update-grub`, **verify the default `grub.cfg` entry**, reboot on `--reboot` - only. The deployer plans nothing under `dankclaw/host/`: kernel state is not a - container restart. -- **Measured arms, byte-identical traffic.** OFF arm at uptime 2 d 23 h: cold - prefill **1,585.1 t/s** (59,024 tok), warm 208,531, cache ratio 131.6×, needle - hit/hit @21,568, decode 37.3 t/s. PT arm after the reboot: **1,616.0 t/s - (+2.0 %)**, warm 214,131, ratio 132.5×, needle hit/hit, decode 33.7 t/s. The - corpus was rebuilt after the reboot deleted `/tmp` (tmpfs) and is provably the - same input: 2,891 lines, **1,287,656 B**, and the engine billed identical token - counts. Verdict: **identity DMA costs nothing on this host** — the 13–16 % that - justified `amd_iommu=off` for five weeks never existed here. Decode's −9.7 % is - instrument noise: three repeats of the same request on the same boot gave 31.8, - 46.9, 47.8 t/s. -- Gates after boot: 0 failed units, systemd running, 32 IOMMU groups (31 identity - + 1 DMA = the Genesys Logic SD reader `c2:00.0`, a kernel quirk), iGPU `c5:00.0` - identity group 20 with ATS, interrupt remapping on. frank-dragoon's ledger is - byte-for-byte the same shape (62.9 + 5.5 + 5.7 = 74.1 GiB, 2 slots over ONE - 200,704 pool, MTP depth 2, prompt cache ON, no clamp; listening at 39.8 s), and - Gitea / Caddy edge / Grafana answered 200. -- **The XDNA2 NPU is back:** `amdxdna` bound `0000:c6:00.1`, firmware - `amdnpu/17f0_11/npu_7.sbin`, `amdxdna_accel_driver 0.7.0 … minor 0`, group 26 - `identity`, PASID width 10. `/dev/accel/accel0` `crw-rw---- root:render 261,0` - with `dank` in `render`. No endpoint deployed — revival needs a quadlet with +- Window executed 01:24–01:35 on Frank's go-ahead, with the A/B the + research demanded. Provenance first: `/etc/default/grub` had never been in + the repo while the IOMMU line was the most consequential kernel decision + on the box, so it is now tracked at `dankclaw/host/grub` (fetched + byte-exact) and moves only through `scripts/set-dankclaw-bootline.sh` — + diff, install 0644 root:root, `update-grub`, **verify the default + `grub.cfg` entry**, reboot on `--reboot` only. The deployer plans nothing + under `dankclaw/host/`: kernel state is not a container restart. +- **Verdict: identity DMA costs nothing on this host.** Byte-identical + traffic, +2.0 % cold prefill going OFF→PT; the 13–16 % that justified + `amd_iommu=off` for five weeks never existed here. Decode's apparent + −9.7 % was instrument noise: three repeats of the same request on the + same boot spread 31.8–47.8 t/s. +- **The XDNA2 NPU is back:** `amdxdna` bound `0000:c6:00.1`, + `/dev/accel/accel0` `crw-rw---- root:render 261,0` with `dank` in + `render`. No endpoint deployed — revival needs a quadlet with `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`. -- Deliberately NOT done: the `pp_dpm_fclk` level-7 pin (it changes iGPU timing for - every workload and nothing co-runs on the NPU yet) and any PerfOpt work (kernel - `7.0.0-38` has zero `perfopt` symbols; the 26.04 archive caps at `7.0.0-38.38`). -- New tool: `scripts/stage-frank-dragoon-eval.sh` rebuilds `/tmp/halogen-eval` - byte-identically (corpus + probe PNG + battery) — run it through `bash -lc`, - `uv` is in `~/.local/bin`. Docs re-keyed: decision record (banner, front matter, - verification/revert, new § *2026-10-18 execution*), `frank-npu.md` (precondition - satisfied, endpoint still absent), AGENTS.md key lesson, both index pages. - bats 371/371. frank-dragoon and frank-zealot were live the whole window; only - the reboot interrupted them (~40 s to listen, health 200). +- Deliberately NOT done: the `pp_dpm_fclk` level-7 pin (it changes iGPU + timing for every workload and nothing co-runs on the NPU yet) and any + PerfOpt work (kernel `7.0.0-38` has zero `perfopt` symbols; the 26.04 + archive caps at `7.0.0-38.38`). +- Gates after boot: 0 failed units, 32 IOMMU groups (31 identity + 1 DMA = + the Genesys Logic SD reader, a kernel quirk), interrupt remapping on; + frank-dragoon's ledger byte-for-byte the same shape and Gitea / Caddy + edge / Grafana answered 200. New tool `scripts/stage-frank-dragoon-eval.sh` + rebuilds the A/B corpus byte-identically — run it through `bash -lc`, `uv` + is in `~/.local/bin`. Docs re-keyed: the decision record + `dankclaw-iommu-translation-off.md` (new § *execution*), `frank-npu.md` + (precondition satisfied, endpoint still absent), AGENTS.md key lesson. ## 2026-10-18 — frank-zealot: Bonsai-2-9B PQ2_0 with the MTP head (3 × 262,144) - -- Swapped `prism-ml/Ternary-Bonsai-2-27B-gguf` → **`Ternary-Bonsai-2-9B-gguf`** - at PQ2_0 (`Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, 2,675,410,720 B, Hub-LFS - sha256 `c79fcdfaf6b7…` verified at fetch) plus the publisher's **Q8_0** mmproj - (624,229,664 B, `e0cb6f69…`), into `/home/dank/models/Bonsai2-9B-MTP/`; the - runtime moved to vendor fork build **10777** (`e3de71522`) because the MTP - arm needs the build that pairs MTP with a quantized V cache. -- The win is the **window, not the quant**: `--ctx-size 786432 --parallel 3` - puts every conversation back at the trained 262,144 (the 27B era sold each - slot 200,704), and the fleet's widest context makes :7771 the **compaction - hub** — `models.yml` `contextWindow` 200704 → 262144. Measured vs the 27B: - decode 167.2 vs 76.6 t/s single-stream, aggregate 280.2 vs 109.4 t/s across - the (now three) concurrent streams, prefill 2,877 t/s at 106,418 depth. -- **MTP armed, then OFF by measurement.** The head is in the file, the fork - loads it, and the draft context bills its own **f16** KV over the WHOLE pool - — 3,072 MiB at 786,432 positions, ledger-confirmed. At `UBATCH_SIZE=2048` - that squeeze **halved decode (167.2 → 109.2 t/s)**. The documented MTP shape - is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s; declined, because - +17 % decode is not worth spending 6.2 GiB of the 6,249 MiB free. The arm is - recorded in the env header, not left to rediscovery. -- Two bugs the trial caught, both fixed in the repo: `${SPEC_PRESENCE_PENALTY:=0.0}` - was a *default-assignment*, so the env value always won and the documented - "MTP runs without `--presence-penalty`" arm was unreachable — now `${VAR=0.0}`, - which lets `SPEC_TYPE=draft-mtp` drop the flag. And `up{job="frank-zealot"}` - had read 0 for the **entire retained Prometheus history**: llama.cpp answers - **501** on `/metrics` unless `--metrics` is passed, so the scrape failed beside - a healthy `/health` and `LlmEndpointDown` fired for days. `METRICS=1` is now - the default and the seam pins it. -- Boot ledger: KV `13,056.00 MiB` (8 full-attention layers × 4 KV heads × 256 - dim, q8_0 K + q8_0 V over 786,432 positions — ledger-exact), settled - **17,876 used / 6,249 free**; the f16 pair would need 26,112 MiB and cannot - boot this card. `TimeoutStopSec=180` carries over (SIGTERM ignored past the - old default during the 03:38 stop). -- Header census that corrected my own first pass: `block_count` is **33** - (32 trunk + the `blk.32` MTP block, which carries a full attention + FFN in - q8_0 — that is why the draft context bills real KV), **202 of 442** tensors - carry the fork's PQ2_0 type (norms f32, MTP projections q8_0), `hadamard` - `block_size` **4096** with `gdn_grouped=True`. The "2.13 bpw / type 142 / 32 - layers" figures inherited from the 27B page were wrong for this file and are - corrected in AGENTS.md, the env header and the runbooks. -- Docs: new runbook `LLM/frank-zealot-llama-prism-bonsai9-mtp.md`; the 27B page - re-banned **RETIRED 2026-10-17 → 10-18** with its bench-window section closed; - both index pages, the homepage card, `Dankclaw/OpenWebUI.md` and the live - picker row (DB `description` + context length 262,144) re-keyed. Rollback: - revert the swap commit — the 27B payload and the build-10754 fork tree are - both still on disk. -- Deployed 13:16 PDT (`deploy.sh dankclaw --service frank-zealot`, e2e 33/33) - — and the first boot was **not the shipped payload**: a leftover - `frank-zealot.service.d/ptq1.conf` from the bench window (written 03:29 for - the 27B **PTQ1_0** arm Frank ordered at 02:36 and measured from 03:14 — see - the record below) pointed - `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit booted - that arm: 3 slots of 133,888, 21,188 MiB used, batch 3072/ubatch 1536. - `/health` 200, `/v1/models` echoed the alias, Playwright 33/33 — nothing in - the gate reads the payload. Drop-in moved to - `~/.AGENTTRASH/20261018-zealot-ptq1-arm/`, `daemon-reload`, restart at 13:19: - `n_slots = 3, n_ctx_slot = 262144`, KV 13,056.00 MiB, settled 17,922/6,203. -- Production measures (engine `timings`, greedy, `ignore_eos`): **168.7 t/s** - decode (168.4/168.7/168.7) and **~300 t/s** aggregate across three 512-token - streams (102.8 each). My first battery reported 117 and 72 t/s: it timed - natural generations that stopped at 24 tokens — wall-clock aggregate on an - early EOS reads 4× low. `ignore_eos` + the engine's own numbers is the only - honest way to time this. -- Ledger correction: the shipped shape prints **RS 150.75 MiB** (3 cells), not - the 452.25 MiB my first-pass notes carried — that figure belongs to the - SPEC-ARMED arms, where MTP depth 2 bills three seqs per slot (9 cells). The - drafter triples the recurrent state as well as billing 3,072 MiB of f16 KV. - Corrected in AGENTS.md (topology row + architecture section), the env header - and the runbook, with the production boot ledger replacing the trial's. -- Verified live: chat `27*59` → 1593; `where_am_i` tool call fired; 512 px - solid-red PNG → **Red** (278 prompt tokens, `process_mtmd` in the journal); - `/props` `n_ctx 262144`, `presence_penalty 1.0`; `/metrics` 200 with 15 - `llamacpp:*` families and `up{job="frank-zealot"} = 1` — the first non-zero - sample in that job's retained history. -- OpenWebUI `model` rows re-keyed in the live DB (backup - `webui.db.bak-20261018`): `frank-zealot` → 262,144 + the 9B description, and - `frank-dragoon`'s `context.length` corrected 262,144 → **200,704** (it had - kept the pre-lowering window from 2026-10-17 while its own description said - 200,704). Note the columns are `params`/`meta` JSON — the table has no - `context_length`/`description` columns. +- 2026-10-18 — frank-zealot: 27B PQ2_0 → **Ternary-Bonsai-2-9B PQ2_0 with + the MTP head**, runtime moved to the PrismML vendor fork build **10777** + because the MTP arm needs the build that pairs MTP with a quantized V + cache. See [[frank-zealot]]. +- The win is the **window, not the quant**: every conversation goes back at + the trained 262,144 (the 27B era sold each slot 200,704), and the fleet's + widest context makes :7771 the **compaction hub** — `models.yml` + `contextWindow` 200704 → 262144. +- **MTP armed, then OFF by measurement.** The head is in the file and the + fork loads it, but the draft context bills its own **f16** KV over the + WHOLE pool, and at `UBATCH_SIZE=2048` that squeeze **halved decode**. The + documented MTP shape (`SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024`) was + declined — the decode gain is not worth spending the free card on it. The + arm is recorded in the env header, not left to rediscovery. +- Two bugs the trial caught, both fixed in the repo: + `${SPEC_PRESENCE_PENALTY:=0.0}` was a *default-assignment*, so the env + value always won and the documented "MTP runs without + `--presence-penalty`" arm was unreachable — now `${VAR=0.0}`, which lets + `SPEC_TYPE=draft-mtp` drop the flag. And `up{job="frank-zealot"}` had read + 0 for the **entire retained Prometheus history**: llama.cpp answers + **501** on `/metrics` unless `--metrics` is passed, so the scrape failed + beside a healthy `/health` and `LlmEndpointDown` fired for days. + `METRICS=1` is now the default and the seam pins it. +- Deployed 13:16 (`deploy.sh dankclaw --service frank-zealot`) — and the + first boot was **not the shipped payload**: a leftover + `frank-zealot.service.d/ptq1.conf` from the bench window pointed + `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit + booted that arm. `/health` 200, `/v1/models` echoed the alias, Playwright + 33/33 — **nothing in the gate reads the payload.** Drop-in moved to + `~/.AGENTTRASH/`, `daemon-reload`, restart: the shipped trio booted. (The + drop-in was not a rogue — see the PTQ1_0 entry below.) +- Measurement law: the first battery reported 117 and 72 t/s because it + timed natural generations that stopped at 24 tokens — wall-clock aggregate + on an early EOS reads 4× low. `ignore_eos` + the engine's own `timings` + numbers is the only honest way to time this endpoint. +- OpenWebUI `model` rows re-keyed in the live DB. Note for the next edit: + the values live inside the `params`/`meta` JSON columns — the table has no + `context_length`/`description` columns — and there is no `sqlite3` binary + on the host, so `python3` stdlib + `Connection.backup()`. ## 2026-10-18 — frank-zealot PTQ1_0 arm: measured, approved, and never written down - - **Correction to the entry above and to my own report.** The - `frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished experiment. - The session archive (`01a14c93`) carries Frank's own instruction at **02:36**: - replace frank-zealot with `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8** - mmproj, batch/ubatch **3072/1536**, "fit as much context as possible in 2 - contexts". -- It was the production payload **03:14:39 → 03:38:57** (pid 191358; journal - `type ptq1_0: 402 tensors`, `file type = PTQ1_0 — 1.75 bpw ternary`, mmproj - loaded, 2 × 200,704) and served real traffic. Recovered from that process's - own `print_timing` lines (17 timed tasks): single-stream greedy decode - **48.0 and 47.6 t/s at 512 tokens, 47.9 at 229, 49.0 at 451**; 25–27 t/s per - stream with two streams live; prefill **1,295 t/s @ 30,637 → 943 @ 97,856 → - 801 t/s @ 141,365** (one 176 t/s row at 97,024 is a concurrent-slot request, - not a clean probe). -- Against the PQ2_0 27B record (76.6 t/s decode, 691.7 t/s prefill at 166k): - PTQ1_0 freed the **1,201 MiB** its fetch script promised and paid **−37 % - decode** — the publisher's "dense ternary unpacking loses on Ampere" warning, - now confirmed on GA102 — while prefilling deep prompts faster than that - record ever managed. Frank scratched the whole 27B plan at **03:33** ("scratch - the current frank-zealot plan completely… 9B… enable mtp… if you can possibly - fit a third context, do it"), which is what shipped at 13:16. + `frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished + experiment. The session archive (`01a14c93`) carries Frank's own + instruction at **02:36**: replace frank-zealot with + `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8** mmproj, batch/ubatch + **3072/1536**, "fit as much context as possible in 2 contexts". +- It was the production payload **03:14:39 → 03:38:57** and served real + traffic; recovered from its own `print_timing` lines, PTQ1_0 ran + ≈47.6–49.0 t/s single-stream greedy decode (25–27 t/s per stream with two + streams live) and prefilled deep prompts faster than the PQ2_0 27B record + ever managed. +- Against the PQ2_0 27B record: PTQ1_0 freed the **1,201 MiB** its fetch + script promised and paid **−37 % decode** — the publisher's "dense + ternary unpacking loses on Ampere" warning, now confirmed on GA102. + Frank scratched the whole 27B plan at **03:33** ("scratch the current + frank-zealot plan completely… 9B… enable mtp… if you can possibly fit a + third context, do it"), which is what shipped at 13:16. - The durable failure is mine: those numbers lived only in `/tmp` and the - journal, never in the repo, so the next agent on the host could not tell an - approved payload from a rogue drop-in and called it a hijack. The runbook - deploy ledger now carries the measurement; `/home/dank/models/Bonsai2-PTQ1_0/` - stays on disk as the 27B rollback and the drop-in stays trashed. - + journal, never in the repo, so the next agent on the host could not tell + an approved payload from a rogue drop-in and called it a hijack. The + fetch script's header now carries the measurement; + `/home/dank/models/Bonsai2-PTQ1_0/` stays on disk as the 27B rollback and + the drop-in stays trashed. ## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot -- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33. The - drop-in check the zealot incident bought ran FIRST: `DropInPaths` empty, so - the repo env governed. Boot ledger: `reserving the KV pool, 401408 positions - (about 11.0 GiB)`, `2 slots over ONE 401408-position KV pool, each request up - to 200704`, and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of - working memory, 79.6 GiB in all`; prompt cache ON (12 entries × 111 MiB at - ctx 200704 = 1.3 GiB), n-gram 47.68 GiB still `Pin::None`, no clamp line. -- Warm reuse survived the raise, measured right after the deploy: one - **101,987**-token wikitext prompt through `/v1/chat/completions` took - **81.5 s** cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats — - `HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn - prefills nothing and the doubling evicted nothing the cache needed. The RAM - price is still real: host `MemAvailable` fell **26 → 22 GiB**, because every - pool position is RAM the n-gram table's page cache loses. +- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33, and + the drop-in check the zealot incident bought ran FIRST: `DropInPaths` + empty, so the repo env governed. 2026-10-18 — frank-dragoon: the KV pool + doubled to 401,408 positions over the 200,704 ceiling; the measured boot + ledger owns the shape. See [[frank-dragoon]]. +- Warm reuse survived the raise: one **101,987**-token prompt took 81.5 s + cold and 0.3 s on repeats — `HALOGEN_CACHE_INPLACE=1` keeps the KV rows in + the pool, so a warm turn prefills nothing and the doubling evicted nothing + the cache needed. The RAM price is still real: host `MemAvailable` fell + **26 → 22 GiB**, because every pool position is RAM the n-gram table's + page cache loses. - Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the - engine reserves **11.0 GiB** and its preflight bills **11.3**. Re-keyed in - AGENTS.md (topology row, KV-pool law, memory ledger), the `frank-dragoon.env` - lore block, both index pages, the iommu page and the runbook, and every - "committed but not yet deployed" clause was retired with the measurement. -- The same sweep caught the LLM index still describing frank-zealot as - **Bonsai-2-27B at 2 × 200,704** in its intro paragraph and topology diagram — - the 9B swap of the same day had not reached those two spots. Re-keyed: 8 - full-attention + 24 Gated DeltaNet + the `blk.32` MTP block, 3 × 262,144, - tensor type **141**. -- PTQ1_0 provenance corrected (see the dedicated section above): the arm that + engine reserves **11.0 GiB** and its preflight bills **11.3**. The measured + figure wins, and every "committed but not yet deployed" clause was retired + with the measurement (AGENTS.md, the env lore block, both index pages, the + iommu page, the runbook). +- PTQ1_0 provenance corrected (see the dedicated entry): the arm that hijacked the zealot deploy was a **repo-tracked, Frank-ordered, measured** - candidate — `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and - its header now carries the recovered numbers (≈48 t/s decode, 801 t/s prefill - at 141k) and the verdict: −37 % decode against PQ2_0 for 1,201 MiB freed, then - scratched by Frank's 03:33 order in favour of the 9B. Nothing is owed on it. -- OpenWebUI: both decoration rows re-keyed in the live DB. The values live - inside JSON columns — `params.context.length` and `meta.description` — not in - fields named for them, and there is no `sqlite3` binary on the host, so - `python3` stdlib + `Connection.backup()` (`webui.db.bak-20261018`). Zealot - 200,704 → **262,144** with the 9B description; dragoon 262,144 → **200,704**, - which had kept the pre-lowering window while its own description said - 200,704. + candidate — `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, + and its header now carries the recovered numbers (≈48 t/s decode, 801 t/s + prefill at 141k) and the verdict: −37 % decode against PQ2_0 for 1,201 + MiB freed, then scratched by Frank's 03:33 order in favour of the 9B. + Nothing is owed on it. - Census of the halogen HTTP surface while doing it, because the fleet bench - law quoted the wrong engine: frank-dragoon answers **`/health`, `/metrics`, - `/v1/models`, `/v1/chat/completions`(POST)** and 404s `/ping`, `/props`, - `/completion`, `/tokenize`; the chat response carries **no `Server-Timing`** - header (`server: uvicorn`, only `date`/`content-*`). So the `/completion` - `timings` object and the `ttft;dur=` header are **llama.cpp** instruments — - frank-zealot's, not dragoon's. AGENTS.md's Benchmarking Policy re-keyed to - say which engine owns which instrument. + law quoted the wrong engine: frank-dragoon answers `/health`, `/metrics`, + `/v1/models`, `/v1/chat/completions`(POST) and 404s `/ping`, `/props`, + `/completion`, `/tokenize`; the chat response carries **no + `Server-Timing`** header. So the `/completion` `timings` object and the + `ttft;dur=` header are **llama.cpp** instruments — frank-zealot's, not + dragoon's. AGENTS.md's Benchmarking Policy re-keyed to say which engine + owns which instrument. ## 2026-10-18 — frank-tts phase 1: the cloned-voice endpoint, named and tested (not deployed) - - Voice-over goal, phase 1 (Gitea issue #2). New service **frank-tts** on - Dankclaw: `dankclaw/voiceservice/` (stdlib-only `app.py` + `Containerfile` + - rootless quadlet), `scripts/frank-tts.env`, `tests/frank_tts_unit.bats` - (12 tests). One OpenAI-shaped `POST /v1/audio/speech` surface; `voice=` - is honoured only when `/voices//reference.wav` exists, otherwise a 400 - listing the voices that are — no silent fallback voice. `voice_id` is the - harness alias; `response_format=mp3` really is mp3; oversized input is a 414, - not a half-narrated sentence; `/health` reports backend + voice list; no audio - ever reaches the log. Two backends behind it (`TTS_BACKEND=omni-http` forwards - to the host's `:8880` engine with the voice as base64 `reference_audio`; - `llama-tts-cli` runs a binary per request at `nice -n 19`), both exercised - through stubs — no GPU, no containers in the seam. + Dankclaw: `dankclaw/voiceservice/` (stdlib-only `app.py` + `Containerfile` + + rootless quadlet), `scripts/frank-tts.env`, `tests/frank_tts_unit.bats`. + One OpenAI-shaped `POST /v1/audio/speech` surface; `voice=` is + honoured only when `/voices//reference.wav` exists, otherwise a 400 + listing the voices that are — **no silent fallback voice**; oversized + input is a 414, not a half-narrated sentence; `/health` reports backend + + voice list; no audio ever reaches the log. - **Naming law applied (Frank's call):** treat it as a first-class model endpoint → alias-named `frank-tts` (TTS endpoint, so not `frank-asr`), - **port 7772**, joining `7770 frank-dragoon` / `7771 frank-zealot`; the engine - upstream keeps 8880 and is deliberately not a fleet endpoint. The internal - knob names stay `SHIM_*` (they name the façade component, not the service). -- **Voice-pack law recorded** in the env header: a pack needs `reference.wav` - (24 kHz mono, 18–25 s) **and** the pinned engine's own speaker artifact — - required, beside the WAV, never replacing it — plus `manifest.json` provenance - whose offsets are always past the first 10 s of the source (the - source-material law). Enforcement is split honestly: the shim gates on - `reference.wav` (the only engine-agnostic term); the pack builder is what will - refuse to emit a pack without its artifact. -- Suite green: **384/384** (`./tests/run.sh`, wall 66 s). Landed as **PR #4** - (branch `wt/20261017-181502` → `main`, merged as `49c4e19`) — the first PR on - this repo through the API, which cost three 422s to get right: the merge - endpoint takes **`{"Do":"merge"}`**, `Do` being a *string* naming the strategy. - `DoMerge`/`merge_method` do not exist on this Gitea; swagger JSON is disabled - on the instance, so the 422 messages are the schema. Law saved to the - `gitea-api-pr-operations` skill. -- Deliberately NOT in this phase: the `:8880` engine decision and its - speaker-artifact format, the pack builder, deployer/homepage/Prometheus - registration, the `models.yml` row that lets omp call it, and open-notebook. - Harness fact established while writing the PR body: the **`tts` tool** will - dispatch an `api: openai-speech` provider row to `POST /audio/speech` - (`{model, input, response_format, voice}`), but the live `speech.*` vocalizer is - Kokoro-enum-only — so "the agent talks in my voice" needs that provider row or - a `pi.registerProvider` extension, decided in phase 2. + **port 7772**, joining `7770 frank-dragoon` / `7771 frank-zealot`; the + engine upstream keeps 8880 and is deliberately not a fleet endpoint. The + internal knob names stay `SHIM_*` (they name the façade component, not the + service). +- **Voice-pack law recorded** in the env header: a pack needs + `reference.wav` (24 kHz mono, 18–25 s) **and** the pinned engine's own + speaker artifact — required, beside the WAV, never replacing it — plus + `manifest.json` provenance whose offsets are always past the first 10 s of + the source (the source-material law). Enforcement is split honestly: the + shim gates on `reference.wav` (the only engine-agnostic term); the pack + builder is what will refuse to emit a pack without its artifact. +- Landed as **PR #4** (branch `wt/20261017-181502` → `main`, merged as + `49c4e19`) — the first PR on this repo through the API, which cost three + 422s to get right: the merge endpoint takes **`{"Do":"merge"}`**, `Do` + being a *string* naming the strategy — `DoMerge`/`merge_method` do not + exist on this Gitea, and swagger JSON is disabled on the instance, so the + 422 messages are the schema. Law saved to the `gitea-api-pr-operations` + skill. ## 2026-10-18 — TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion - - Frank listened to the four audition pairs (`qwentts-*.mp3` against their - `armB-*.mp3` mainline partners) and **passed every clone**. That deleted the - only axis the A/B could not measure, and the question was reframed: **which - engine is better for live output**. On that criterion RTF only has to stay - under 1.0 with margin — the discriminators are time-to-first-audio and - run-to-run spread, because an arm whose RTF can cross 1.0 stalls a live - stream mid-sentence. -- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, 1.7 B - talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on 8880.** - The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` stays in the - repo as the rollback. The numbers that carried it: first audio byte 86–451 ms - against a per-request model load, warm RTF 0.55–0.67 against 0.93–0.99, spread - ±0.01 against ±0.25, and a native `/v1/audio/speech` against no HTTP path at - all. Three consequences are configuration, not preference: boot-warm every - voice, do not pin a cpuset (threads are `hardware_concurrency()/2`), register - every pack with its WAV (a lone embedding is rejected). -- Two follow-ups deliberately **not** decided: the **0.6 B talker** (0.44 RTF, - 3.4 GiB, never auditioned — every clip is 1.7 B) and **ICL vs - embedding-only** conditioning. Embedding-only is the shipped default because - it is cheaper and needs no `ref_text`, which this fleet cannot transcribe - honestly; `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice + `armB-*.mp3` mainline partners) and **passed every clone**. That deleted + the only axis the A/B could not measure, and the question was reframed: + **which engine is better for live output**. On that criterion RTF only has + to stay under 1.0 with margin — the discriminators are time-to-first-audio + and run-to-run spread, because an arm whose RTF can cross 1.0 stalls a + live stream mid-sentence. +- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, + 1.7 B talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on + 8880.** The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` + stays in the repo as the rollback. Three consequences are configuration, + not preference: boot-warm every voice, do not pin a cpuset (threads are + `hardware_concurrency()/2`), register every pack with its WAV (a lone + embedding is rejected). +- Two follow-ups deliberately **not** decided: the **0.6 B talker** (never + auditioned — every clip is 1.7 B) and **ICL vs embedding-only** + conditioning. Embedding-only is the shipped default because it is cheaper + and needs no `ref_text`, which this fleet cannot transcribe honestly; + `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice opt-in for batch work where total time beats first byte. -- **Addressing law corrected** in - [[Local TTS and Voice Cloning]]: the pair may not meet at a host-loopback - publish. netavark DNATs published ports only and a bridge container cannot - reach a host `127.0.0.1` bind — the lesson that 502'd hermes — so - `host.containers.internal:8880`, which `scripts/frank-tts.env` documents, is - correct only for a routable bind. The deployed pair shares one quadlet network - and the façade addresses the engine by container name. -- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`, 8 commits, 403 tests) is - **merged** as `1eb8b21` (Frank's OK, given in session): open-notebook + - SurrealDB quadlets on the private bridge `10.89.4.0/24`, the API published on - **5056** because seerr holds `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT` - moved 600 → **660** to sit one round-trip outside frank-tts's own - `SHIM_TIMEOUT`. Merged is not runnable: nothing listens on 7772 until the - voice stack lands, and the episode destination (`/home/dank/media/podcasts`) - stays interim pending Gitea #7. -- Still in flight, not in this branch: `feat/frank-tts-engine` — the arm A image - build, the quadlet pair on one network, the boot-warm step, deployer / - planner / `test-changed.sh` routing, homepage and docs. This branch is also - where the two-branch overlap got resolved: merging `main` here conflicted on - `log.md` alone (both branches appended an entry) and took `index.md`, - `deploy-remote.sh`, `deploy-plan.py` and `test-changed.sh` clean. +- **Addressing law corrected** in [[Local TTS and Voice Cloning]]: the pair + may not meet at a host-loopback publish. netavark DNATs published ports + only and a bridge container cannot reach a host `127.0.0.1` bind — the + lesson that 502'd hermes — so `host.containers.internal:8880`, which + `scripts/frank-tts.env` documents, is correct only for a routable bind. + The deployed pair shares one quadlet network and the façade addresses the + engine by container name. +- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`) is **merged** as + `1eb8b21` (Frank's OK): open-notebook + SurrealDB quadlets on the private + bridge `10.89.4.0/24`, the API published on **5056** because seerr holds + `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT` moved 600 → **660** to sit + one round-trip outside frank-tts's own `SHIM_TIMEOUT`. Merged is not + runnable: nothing listens on 7772 until the voice stack lands, and the + episode destination stays interim pending Gitea #7. ## 2026-10-18 — open-notebook: the podcast generator's config lands (nothing deployed) -- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — notes - and sources in, a multi-speaker episode out, script written by +- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — + notes and sources in, a multi-speaker episode out, script written by **frank-zealot**, voices rendered by **frank-tts**, no cloud provider - anywhere in the path. Repo-side only: no host touched, no container pulled or - started. Runbook: [[Open Notebook]]. + anywhere in the path. Repo-side only: no host touched. Runbook: + [[Open Notebook]]. - **Two containers, not one.** `lfnovo/open_notebook:v1-latest` plus - `surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` and a - documented one-time `podman pull` — frank-dragoon's pinning style, because - `:latest` moving underneath a rootless service is how you get an unbootable - pod after an unrelated `systemctl-daemon-reload`. The registry round-trip is - what proved `v2` is SurrealDB **2.7.0** and that the app image is Debian - trixie with **curl and ffmpeg already inside** — which is how both health - probes can be argument-only: `curl -fsS …/health` (route is upstream's, - `api/main.py:239`, and it sits on the auth exclude list) and - `/surreal is-ready`, which I ran against the digest here: green in 2 s, and - it refuses to pass without `SURREAL_PASS` in the environment — a credential - that must never ride the command line, so `Exec=` carries flags only. -- **Host 5055 was already spoken for.** seerr has held `127.0.0.1:5055` since - 2026-10-14 (live `ss -ltn` agrees today), so the API publishes - `0.0.0.0:5056:5055`. open-notebook's frontend derives its API URL from the - window origin when `API_URL` is unset — the upstream env reference spells the - reverse-proxy symptom — so a silent remap would hang the UI on a port nobody - listens on, and `API_URL=http://192.168.0.143:5056` is set accordingly. 8502 - publishes on `0.0.0.0` because this service carries its own password (the - episode-audio route is unauthenticated), unlike the loopback-and-proxied - observability stack; **8000 is published nowhere**, where upstream exposes it - on loopback with root credentials, and the seam forbids it. -- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread engine - per request on a box that also serves two LLMs, while open-notebook defaults - to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way - self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the - worker knob is only real because the image's own supervisord expands - `${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). The TTS clock - goes to **660 — one round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so - a shim give-up reads as the shim's error rather than the worker's; the seam - asserts the direction (`>`) against the frank-tts env instead of restating a - number. `ESPERANTO_LLM_TIMEOUT` stays at the app's 180 s default: frank-zealot - answers an outline or a transcript well inside it, and upstream says to keep it - under 600 so a slow call fails with a name on it. -- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY` - must be set before the UI can save *any* provider credential, and - `OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app - reads `os.environ.get(…) is not None`, so `PASSWORD=` boots a login form - asking for the empty string. The repo env file carries both keys empty; the - deployer seeds `/home/dank/Containers/open-notebook/env` (600) from the host - vault and **refuses to start the pair while either secret is blank**, because - a half-configured open-notebook stores no credentials and fails quietly. -- **Episode destination is interim, by decision.** The destination is **Gitea - issue #7, `human-only`** — agents must not work it. Until Frank settles it, - the app keeps its own named volume and `/home/dank/media/podcasts` is - bind-mounted over `/app/data/podcasts` (`DATA_FOLDER` is hard-coded `./data`), - so finished MP3s are ordinary files in his home directory instead of volume - contents needing `podman unshare` — the database volume demonstrably lands on - a mapped subuid, measured with the pinned image. One mount line; one deletion - undoes it. -- **Two UI buttons will fail, and that is cosmetic.** Test Connection does - `GET /models` and the TTS probe synthesises voice `alloy`; frank-tts - serves `/health` and `/v1/audio/speech` only. Model rows are created *by - name* and the voice field is free text, so `narrator1/2/3` is typed in and - never discovered. Documented, not worked around. -- **Blocked on frank-tts, stated plainly:** phase 1 landed without deployer or - homepage registration and nothing listens on 7772 today, so a generation run - writes its script and then fails on the first voice. That is upstream's - creation-podcasts warning in fleet clothing. -- Suite green: **403/403** (`./tests/run.sh`, wall 67 s) — 19 tests in the new - `tests/open_notebook.bats`, structure only: pins, boot persistence, the - unpublishable database port, the 5055→5056 remap plus the `API_URL` that has - to match it, the env-file read order, the guard knobs, the health gate naming - `/health`, planner routing, the homepage card hidden while undeployed. + `surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` — + frank-dragoon's pinning style, because `:latest` moving underneath a + rootless service is how you get an unbootable pod after an unrelated + `systemctl-daemon-reload`. Both health probes are argument-only + (`curl -fsS …/health`, `/surreal is-ready`), and `is-ready` refuses to + pass without `SURREAL_PASS` in the environment — a credential that must + never ride the command line, so `Exec=` carries flags only. +- **Host 5055 was already spoken for** — seerr has held `127.0.0.1:5055` — + so the API publishes `0.0.0.0:5056:5055`, and because open-notebook's + frontend derives its API URL from the window origin when `API_URL` is + unset, `API_URL=http://192.168.0.143:5056` is set to match: a silent + remap would hang the UI on a port nobody listens on. 8502 publishes on + `0.0.0.0` because this service carries its own password (the + episode-audio route is unauthenticated); **8000 is published nowhere**, + where upstream exposes it on loopback with root credentials, and the seam + forbids it. +- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread + engine per request on a box that also serves two LLMs, while + open-notebook defaults to a 25-way self-DoS; both knobs are pinned to + `1` in `scripts/open-notebook.env`. The TTS clock goes to **660 — one + round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so a shim + give-up reads as the shim's error rather than the worker's; the seam + asserts the direction (`>`) against the frank-tts env instead of + restating a number. +- **What a credential needs in order to exist:** + `OPEN_NOTEBOOK_ENCRYPTION_KEY` must be set before the UI can save *any* + provider credential, and `OPEN_NOTEBOOK_PASSWORD` must stay **absent** to + remain auth-off — the app reads `os.environ.get(…) is not None`, so + `PASSWORD=` boots a login form asking for the empty string. The deployer + seeds `/home/dank/Containers/open-notebook/env` (600) from the host vault + and **refuses to start the pair while either secret is blank**, because a + half-configured open-notebook stores no credentials and fails quietly. +- **Episode destination is interim, by decision.** The destination is + **Gitea issue #7, `human-only`** — agents must not work it. Until Frank + settles it, the app keeps its own named volume and + `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` + (`DATA_FOLDER` is hard-coded `./data`), so finished MP3s are ordinary + files in his home directory instead of volume contents needing + `podman unshare`. One mount line; one deletion undoes it. +- **Two UI buttons will fail, and that is cosmetic:** Test Connection does + `GET /models` and the TTS probe synthesises voice `alloy`; + frank-tts serves `/health` and `/v1/audio/speech` only. Model rows are + created *by name* and the voice field is free text, so `narrator1/2/3` is + typed in and never discovered. Documented, not worked around. - Three deployer seams cost time and are now law-shaped: a service must be registered in **five** places — `SERVICE_ORDER`, `DANKCLAW_SERVICES`, the `deploy-remote.sh` dispatch case, its pre-deploy requirement, and - `deploy.sh`'s dispatch regex, where an omission silently plans an empty set - for `--only`; a `dankclaw//*` case must precede the `dankclaw/*.container` - glob because that `*` crosses `/`; and the changed-path routing lives *inside* - the per-path loop, after the `scripts/*.env)` case, not at the top of the file. -- Deliberately NOT done: the deploy (pulling 1.27 GiB to Dankclaw was not - asked for), any Caddy vhost or public exposure, audiobook/EPUB (issue #5), any - `frank-tts` code change, and the harbor templates on Dankserver — - `~/.harbor/compose.opennotebook.yml` pins the deprecated `-single` image, - carries no encryption key and injects the whole harbor env — left untouched, - read-only, as the record of what not to copy. + `deploy.sh`'s dispatch regex, where an omission silently plans an empty + set for `--only`; a `dankclaw//*` case must precede the + `dankclaw/*.container` glob because that `*` crosses `/`; and the + changed-path routing lives *inside* the per-path loop, after the + `scripts/*.env)` case, not at the top of the file. ## 2026-10-18 — frank-tts-engine: the arm A engine becomes deployable (nothing deployed) - -- Goal: turn the engine the day's A/B picked into a **repo-owned, boot-persistent, - self-healing container** — `frank-tts-engine`, `qwentts.cpp` commit - `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`, resident `tts-server`, 1.7 B base - talker Q4_K_M + Q8_0 tokenizer, container port 8880, and **not** a fleet - endpoint: the endpoint stays `frank-tts` on 7772. Runbook: [[Frank TTS Engine]]. -- **No `PublishPort` at all, which is the shape the merged A/B states as law.** - netavark DNATs published ports only, so a `127.0.0.1:8880` publish is not a path - to a peer container — the façade could never have used it (the trap that 502'd - hermes) — while being the one LAN-reachable surface of an unauthenticated - synthesiser. Both voice containers therefore sit on one committed quadlet - network, `frank-tts-engine.network` at **10.89.5.0/24** (the next free /24 after - open-notebook's 10.89.4.0/24), and the façade addresses the engine by container - name. Diagnostics moved to `podman exec`, including the deployer's health gate - (`wait_container_healthy`, not `wait_for_health`: there is no URL to dial). -- **The boot warmer is a unit hook, not an operator habit.** Cold-voice law: the - first request for a registered voice decodes the whole reference to snapshot - codec state (6.2–8.0 s, first byte 5 530 ms), later ones restore in 0.5 ms; the - registry is memory-only and re-registering does not warm. So - `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh` with - `TimeoutStartSec=900`, and the `-` prefix is what stops a failed warm from - crash-looping the endpoint under `Restart=always`. It logs `cold=` and `warm=` - per voice, so the under-a-second second request is proven on every boot. -- **Thread law recorded before anyone tunes it away:** no `--threads` exists; - `src/backend.h` uses `hardware_concurrency()/2`, so a cpuset pin silently - changes the thread count. Nothing pins this container. +- Goal: turn the engine the day's A/B picked into a **repo-owned, + boot-persistent, self-healing container** — `frank-tts-engine`, + `qwentts.cpp` commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`, resident + `tts-server`, container port 8880, and **not** a fleet endpoint: the + endpoint stays `frank-tts` on 7772. Runbook: [[Frank TTS Engine]]. +- **No `PublishPort` at all, which the merged A/B states as law.** netavark + DNATs published ports only, so a `127.0.0.1:8880` publish is not a path to + a peer container (the trap that 502'd hermes) — while being the one + LAN-reachable surface of an unauthenticated synthesiser. Both voice + containers therefore sit on one committed quadlet network, + `frank-tts-engine.network` at **10.89.5.0/24** (the next free /24 after + open-notebook's 10.89.4.0/24), the façade addresses the engine by + container name, and diagnostics — including the deployer's health gate + (`wait_container_healthy`, there is no URL to dial) — run through + `podman exec`. +- **The boot warmer is a unit hook, not an operator habit.** The registry is + memory-only and the first request for a registered voice decodes the whole + reference, so `ExecStartPost=-/usr/bin/podman exec frank-tts-engine + /app/warm-voices.sh` with `TimeoutStartSec=900`; the `-` prefix is what + stops a failed warm from crash-looping the endpoint under + `Restart=always`, and it logs `cold=` and `warm=` per voice on every boot. +- **Thread law recorded before anyone tunes it away:** no `--threads` + exists; `src/backend.h` uses `hardware_concurrency()/2`, so a cpuset pin + silently changes the thread count. Nothing pins this container. - Image builds from the pinned commit with the **proven** line, verbatim — - `GGML_BLAS=OFF` is forced (this host has `libopenblas.so.0`, no dev headers), - every GPU backend off, `-j 8`, and no `-DGGML_NATIVE` because ggml's default is - `-march=native` for the building host: the image is built where it runs. Base - `ubuntu:24.04` pinned by index digest; two stages, because ggml builds shared - and the RPATH names the build tree. -- Weights fetched by `scripts/download-frank-tts-engine-qwentts.sh` with byte and - LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the deployer - gates on `--verify-only` rather than pulling 1.4 GiB inside a deploy. -- Registered in all five places the open-notebook entry just named as law, plus - the routing case above the `dankclaw/*.container` glob. New seam - `tests/frank_tts_engine.bats` (18 tests): the Quadlet **generator's own output** - for the network/publish/boot/warm shape, the entrypoint's argv driven from the - tracked env file, the warmer against a stub `tts-server` over a real socket - (registration keys, `ref_text` absent, late engine, hung synthesis, name - refusal), the fetcher's verifier, and both routing tables. -- Deliberately NOT done: the deploy (no host touched, no container started, no - image built), any homepage card or Prometheus job for the engine — it publishes - nothing and exposes no metrics, so there is nothing to probe, which is said out - loud in the page rather than papered over. No ICL/`ref_text` work: the packs - keep `reference.spk`/`reference.rvq` so it stays a per-voice opt-in. + `GGML_BLAS=OFF` forced, every GPU backend off, no `-DGGML_NATIVE` because + ggml's default is `-march=native` for the building host: the image is + built where it runs. Weights fetched by + `scripts/download-frank-tts-engine-qwentts.sh` with byte and LFS-oid + checks; the deployer gates on `--verify-only` rather than pulling 1.4 GiB + inside a deploy. ## 2026-10-18 — frank-tts: the 7772 endpoint gets its deploy wiring, and voice naming is written down - Stage 2 of the voice stack: the **façade** in front of the engine is now a - deployable service too. Nothing deployed, no host touched. Runbook: - [[Frank TTS]]. -- **Two attached networks, in a meaningful order.** `dankclaw_default` first — it - owns the default route and is where consumers live (open-notebook dials - `http://frank-tts:7772/v1` by container name) — then the voice pair's private - bridge. `scripts/frank-tts.env`'s `TTS_UPSTREAM` moves from - `host.containers.internal:8880` to `http://frank-tts-engine:8880`, which is the - actual fix for a shape that could never have worked: a bridge container cannot - reach a host-loopback bind, and the engine publishes nothing. + deployable service too. Runbook: [[Frank TTS]]. +- **Two attached networks, in a meaningful order.** `dankclaw_default` + first — it owns the default route and is where consumers live + (open-notebook dials `http://frank-tts:7772/v1` by container name) — then + the voice pair's private bridge. `scripts/frank-tts.env`'s `TTS_UPSTREAM` + moves from `host.containers.internal:8880` to + `http://frank-tts-engine:8880`, the actual fix for a shape that could + never have worked: a bridge container cannot reach a host-loopback bind, + and the engine publishes nothing. - **`Wants=` + `After=` on the engine, never `Requires=`:** the façade still - carries the `llama-tts-cli` rollback backend, which needs no upstream, so an - engine that will not boot must delay this endpoint, not take it down. -- The façade handler installs the **engine's** quadlet pair as well. Not tidiness: - with a referenced `Network=` file absent, the v1 Quadlet generator produces no - container unit at all, and `--only frank-tts` has no other way to obtain it. It - also `mkdir -p /home/dank/voices` as the deploy user, because rootless podman - auto-creates a missing bind source **root-owned** and the endpoint then reports - "no voices" instead of an error. -- **A voice is a lowercase directory name.** `^[a-z0-9][a-z0-9._-]*$`, ≤ 64 chars, - no slash/`..`/space. The shape is the intersection of two rules that do not - agree: `app.py` would accept mixed case, `tts-server` lowercases names — so a - pack named `Narrator1` resolves through the façade and not through the engine, - and the boot warmer refuses such a directory loudly. `narrator1/2/3` are - load-bearing for open-notebook, and `frank-tts-add-cloned-voice` is the skill - that walks the pack → register → warm → prove-a-second-request flow. -- Registered everywhere the open-notebook entry named as law (planner order puts - the engine before the façade, so a warm registry precedes the first synthesis), - plus the homepage card — `wip: down` and hidden, because nothing listens on - 7772 yet. -- **No Prometheus job, on purpose.** The façade exports no `/metrics`, Prometheus 3 - refuses HTML/JSON content types (the `comfyui` and `services` probes are - documented down for exactly that), and `LlmEndpointDown` is scoped - `up{job=~"frank-.*"}` — so a `/health` scrape would page on Signal every few - minutes for a healthy endpoint. That is the frank-archon never-green lesson - pointed the other way, and it is written in the page instead of being quietly - ignored. -- Tests added to the voice seam: the generator's own rendering of the pair (both - attachments, first-network order, one publish, Wants-not-Requires), the - env-file upstream checked as a **network join** rather than a string - comparison, planner separation and order, the façade handler's steps, the - homepage card's gating, and routing that selects both seams for either - deployable's paths. + carries the `llama-tts-cli` rollback backend, which needs no upstream, so + an engine that will not boot must delay this endpoint, not take it down. +- The façade handler installs the **engine's** quadlet pair as well — not + tidiness: with a referenced `Network=` file absent the v1 Quadlet + generator produces no container unit at all — and it `mkdir -p + /home/dank/voices` as the deploy user, because rootless podman + auto-creates a missing bind source **root-owned** and the endpoint then + reports "no voices" instead of an error. +- **A voice is a lowercase directory name.** `^[a-z0-9][a-z0-9._-]*$`, + ≤ 64 chars, no slash/`..`/space. The shape is the intersection of two + rules that do not agree: `app.py` would accept mixed case, `tts-server` + lowercases names — so a pack named `Narrator1` resolves through the façade + and not through the engine, and the boot warmer refuses such a directory + loudly. `narrator1/2/3` are load-bearing for open-notebook. +- **No Prometheus job, on purpose.** The façade exports no `/metrics`, + Prometheus 3 refuses HTML/JSON content types (the `comfyui` and `services` + probes are documented down for exactly that), and `LlmEndpointDown` is + scoped `up{job=~"frank-.*"}` — so a `/health` scrape would page on Signal + every few minutes for a healthy endpoint. That is the frank-archon + never-green lesson pointed the other way, written in the page instead of + quietly ignored. ## 2026-10-18 — voice-pack builder: the 10-second source-material gate becomes code -- Third slice of the voice stack (after the façade and the engine decision): - `scripts/build-voice-pack.sh` turns a source recording plus a crop offset into - a deployable pack — `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the - engine's own `reference.spk` (8,192 B) + `reference.rvq` (5,456–6,864 B) made - from it, and `manifest.json` provenance. Tracked tunables in - `scripts/voice-packs.env`: pack list, per-voice source/offset/duration/pinned - source sha256, extractor path and engine commit, artifact arithmetic. New runbook +- Third slice of the voice stack: `scripts/build-voice-pack.sh` turns a + source recording plus a crop offset into a deployable pack — + `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the engine's own + `reference.spk` + `reference.rvq` made from it, and `manifest.json` + provenance. Tracked tunables in `scripts/voice-packs.env`; new runbook [[Voice Packs]]. -- **The gate is the product.** The project-wide invariant — no voice material - from the first 10 s of a sample file — had only ever been a sentence in - `scripts/frank-tts.env`. An offset inside it now exits 1 before a directory is - created, and the same assertion runs again against the manifest that was just - written, next to the recorded sha256, so a pack cannot carry a forbidden crop - even if the env file is edited into nonsense. `10` stays a constant in the - builder; the env file carries offsets, never the law. -- **The extractor's contract was read, not guessed:** `qwen-codec` in qwentts.cpp - `5916bf7c` takes `--model --talker -i ` - with an optional `--device`, has **no `-o`** and names both artifacts next to - the input with the extension swapped. So the WAV is staged under its final stem - and `reference.wav` — the file the façade gates on — is moved into place - **last**, after the artifacts and the manifest: a failed extractor leaves a - directory that is not a voice. `--force` retires an existing pack whole to - `.AGENTTRASH/` instead of touching bytes; nothing in this script deletes. -- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B per - 12.5 Hz frame (`22 × floor(samples/1920)`) and `enc_dim 2048 × f32` = 8,192 B. - That makes the **effective crop window 19.84–25 s**, narrower than the WAV - law's 18–25 s, and a too-short crop is refused before the extractor runs. An - engine bump that changes either number fails the build by design; the bump - procedure is in [[Voice Packs]]. -- Verified against reality, not against the model: the pinned `narrator1` crop - (98.739–118.636 s of `audiosample.mp3`, sha256 `2299510…`) comes out of a - `--dry-run` at 955,378 B / 19.897 s with 5,456 B of `.rvq` planned — byte-for-byte - the trial's hand-cut numbers — and all three pinned source digests match the - files on disk. -- Seam: `tests/voice_pack_unit.bats` (13 tests) — refusal inside the forbidden - region creates nothing, a legal crop builds the whole pack, `ffprobe` on the - generated reference, manifest round-trip through `jq` with a sha that matches - the source, re-run-without-`--force` leaves every byte alone, `--force` retires - rather than deletes, `--dry-run` needs no extractor, a wrong-sized artifact - from a bumped engine cannot publish a voice, and every crop in the tracked pack - list clears the invariant. `qwen-codec` is stood in by a stub with the real - tool's I/O shape, so no engine, no GPU, no network, no fleet host. Suite green: - **416/416** (`./tests/run.sh`, wall 77 s) — 403 before this branch. -- Deliberately NOT done: no transcript in the manifest (ICL wants a curated - `ref_text`; the trial's machine transcripts were wrong, and the engine - registers WAV-only voices anyway), no 16 kHz variant (arm A is native 24 kHz), - no loudness pass (not in the pack law), and no pack built on the host — the - three sources still live on the workstation and the engine GGUFs have not been - fetched to `/home/dank/models/Qwen3-TTS-GGUF` yet, so the first real run is a - host step. +- **The gate is the product.** The project-wide invariant — no voice + material from the first 10 s of a sample file — had only ever been a + sentence in `scripts/frank-tts.env`. An offset inside it now exits 1 + before a directory is created, and the same assertion runs again against + the manifest that was just written, next to the recorded sha256, so a pack + cannot carry a forbidden crop even if the env file is edited into + nonsense. `10` stays a constant in the builder; the env file carries + offsets, never the law. +- **The extractor's contract was read, not guessed:** `qwen-codec` in the + pinned qwentts.cpp commit has **no `-o`** and names both artifacts next to + the input with the extension swapped. So the WAV is staged under its final + stem and `reference.wav` — the file the façade gates on — is moved into + place **last**, after the artifacts and the manifest: a failed extractor + leaves a directory that is not a voice. `--force` retires an existing pack + whole to `.AGENTTRASH/` instead of touching bytes; nothing in this script + deletes. +- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B + per 12.5 Hz frame and `enc_dim 2048 × f32` = 8,192 B. That makes the + **effective crop window 19.84–25 s**, narrower than the WAV law's 18–25 s, + and a too-short crop is refused before the extractor runs. An engine bump + that changes either number fails the build by design. ## 2026-10-18 — voice stack audit: three faults that no gate could see - -- The voice stack had never been run end to end — no image built, nothing on - 7772 — so the whole path was audited as static code against the tools it - calls, and three faults came out of it. All three are the same species: a - step that **reports success while doing nothing**. +- The voice stack had never been run end to end, so the whole path was + audited as static code against the tools it calls, and three faults came + out of it. All three are the same species: a step that **reports success + while doing nothing**. - **The build would not have happened.** `Containerfile.engine` created its unprivileged user with `adduser`; the pinned ubuntu:24.04 base has no - `adduser` at all. Proven in the base on Dankclaw rather than argued from - memory: `command -v adduser` prints nothing, and the replacement line - `useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts` - answers `uid=10000(tts) gid=999(tts)`. Its `SYS_UID_MAX` warning is cosmetic - and is the price of matching the façade's uid, so the comment says that - before someone "fixes" it by dropping `--system`. -- **The warm pass would not have warmed.** The unit hands its env file to podman - via `--env-file`, which performs no shell parsing and takes the literal - remainder of the line, while `source` strips one quoting layer. `WARM_TEXT` - therefore reached the container with its quotes, the hook interpolated them - into the synthesis body, the JSON was invalid, and every voice would log - "registered, but the warm synthesis failed" beside an `active` unit. Fixed in - the hook — strip exactly one matched pair, only when the inner content also - carries a quote or backslash, then JSON-escape — and the seam now reads - `scripts/frank-tts-engine.env` **both** ways and asserts the engine gets the - same sentence in both shapes. The bug survived the first deploy precisely - because every existing test used `source`. -- **A rebuilt image would not have been rebuilt.** `build_compose_image` skips - `podman build` on a matching `cfhash` label, and that hash covered the - `Containerfile` alone — so editing `app.py` or `tts-engine-warm.sh` (both - `COPY`'d) skipped the build, the deploy reported success, `/health` answered, - and the container served old code. Each case arm now names the files its - `Containerfile` copies and hashes them with it; comfyui/loki/alloy have no - extras, so their labels are byte-unchanged and they do not rebuild here. -- Tests as behaviour, not as greps: the new seam extracts the real - `build_compose_image` from the deployer and drives it against a stub `podman` - that replays per-image label state, asserting the observable question — did - the deploy ask podman to build, or did it skip? — through build, skip, - engine-hook-edit-rebuilds-engine-only, and `app.py`-rebuilds-the-façade. - Reverting `scripts/deploy-remote.sh` turns it red at the warm-hook step, which - is the fault being prevented. Suite green: **443/443** (`./tests/run.sh`, - wall 110 s); each fix commit was left green on its own so the branch bisects. + `adduser` at all — proven in the base on Dankclaw, not argued from memory. + Replacement: `useradd --system --uid 10000 --home-dir /app --shell + /usr/sbin/nologin tts`; its `SYS_UID_MAX` warning is cosmetic and is the + price of matching the façade's uid. +- **The warm pass would not have warmed.** The unit hands its env file to + podman via `--env-file`, which performs no shell parsing and takes the + literal remainder of the line, while `source` strips one quoting layer. + `WARM_TEXT` therefore reached the container with its quotes, the JSON was + invalid, and every voice would log "registered, but the warm synthesis + failed" beside an `active` unit. Fixed in the hook; the bug survived the + first draft precisely because every existing test used `source`. +- **A rebuilt image would not have been rebuilt.** `build_compose_image` + skips `podman build` on a matching `cfhash` label, and that hash covered + the `Containerfile` alone — so editing `app.py` or `tts-engine-warm.sh` + (both `COPY`'d) skipped the build while the deploy reported success and + `/health` answered with old code. Each case arm now hashes the files its + `Containerfile` copies. - Docs carry the lessons where an operator will hit them: a **Locally built images** section in [[deployment]], two new troubleshooting lines in - [[Frank TTS Engine]] (all-voices-cold = quoted `WARM_TEXT`; façade 502 beside - 200 = model loaded ≠ voices registered), and `Voice Packs.md` step 4 - corrected — the warm hook re-registers **every** pack on every run, because - the engine replaces a name it already holds (`tools/tts-server.cpp:148` at - the pinned commit: *"Re-registering a name replaces the previous entry."*), - so a restart is what an engine bump needs and not what a rebuilt pack needs. -- Opened as PR **#14** against `main`. Deliberately NOT done: no deploy (the - first `--service frank-tts-engine` run is a ~10-minute recursive source clone - and a new unit on a host that serves people — Frank's call), no voice pack - built on the host, no Playwright run (this branch publishes no host port and - touches no vhost), and Gitea **#2** phase 1 stays open until 7772 answers. + [[Frank TTS Engine]] (all-voices-cold = quoted `WARM_TEXT`; façade 502 + beside 200 = model loaded ≠ voices registered), and `Voice Packs.md` + step 4 corrected — the warm hook re-registers **every** pack on every run, + because the engine replaces a name it already holds, so a restart is what + an engine bump needs and not what a rebuilt pack needs. ## 2026-10-18 — frank-tts-engine deployed: it builds, it serves, and the warm hook died on the kernel's argument cap - -- Merged as **#14** (`747c665`) and deployed `--service frank-tts-engine`. The - payload took 18 s to fetch for the 1.4 GiB pair (byte count + Hub LFS sha256, - `--verify-only` OK), and the image built clean — the first real proof of the - `useradd` fix: `STEP 3/13: RUN useradd --system --uid 10000 …` passed with only - the documented cosmetic `SYS_UID_MAX` warning, and the unit was `healthy` after - 31 s. +- Merged as **#14** (`747c665`) and deployed `--service frank-tts-engine`; + the image built clean — the first real proof of the `useradd` fix — and the + unit was `healthy` after 31 s. - Then the deploy reported `0 voice(s) warm`: - `/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long`. The hook - handed the registration body to curl as ONE argument; the kernel caps a single - argument at 131,072 bytes (`MAX_ARG_STRLEN`) and a shipped 955 KB - `reference.wav` base64's to about 1.27 MB, so curl never executed and the hook - logged that as "HTTP no response". Fourth silent fault of the voice stack, same - species as the other three: `/health` stayed green throughout. -- Fix: the body goes to a `mktemp` file and curl gets `@file`. The regression test - is the honest shape — the seam's stub engine is a real listener, `curl` is the - real binary, the fixture pack is 300 KB — so it fails with the same kernel error - the host produced (proved red first, green after). Every shipped pack sits well - over the ~96 KB cliff, so no small fixture could ever have seen this. -- Live proof on the deployed container with the three real packs: - `narrator1 cold=1175ms warm=1874ms`, `narrator2 1373/1892`, `narrator3 - 1816/1487`, exit 0, 15.2 s for the whole pass. -- **A trial law did not survive deployment.** Against an empty registry — - register only, then time — the FIRST synthesis of a freshly registered voice - cost **1 883 ms**, the next two **1 283** and **1 226 ms**; a ~110-word request - cost **9.4 s** then **9.8 s**. The trial's 6.2–8.0 s cold / 0.5 ms restore - ladder, which the warmer's own header repeated, is retired: warming buys - registration — mandatory, because the registry is memory-only — plus about - **0.6 s**. Corrected in [[Frank TTS Engine]] law 1 and its troubleshooting - table, in [[Local TTS and Voice Cloning]], and in [[Voice Packs]]. -- Host state, honestly: the engine is `active` with three voices warm, but the hook - fix is not in the image yet, so the next restart re-breaks it until this branch - is merged and re-deployed. `frank-tts` (the 7772 façade) is still undeployed. -- Deliberately NOT done in this window: the `frank-tts` deploy, any Prometheus - probe for the voice pair (neither container exposes `/metrics`; the blackbox - question is still open), and the podcast render — open-notebook is on `main` but - not installed. - -- Sweep afterwards: the retired ladder still lived in six places the first pass - missed — both quadlet files, `scripts/frank-tts-engine.env`, - `scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row, the research - brief's consequences paragraph, and one test comment. All re-keyed to the - measured claim: about **0.6 s**, and the 400-for-every-name reason the pages - should have led with. The trial entry above is left as written — a dated worklog - entry is a record — and this line is the correction that points at it. + `/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long`. The + hook handed the registration body to curl as ONE argument; the kernel caps + a single argument at 131,072 bytes (`MAX_ARG_STRLEN`) and a shipped ~955 KB + `reference.wav` base64's past that, so curl never executed and the hook + logged that as "HTTP no response". Fourth silent fault of the voice stack, + same species as the other three: `/health` stayed green throughout. Fix: + the body goes to a `mktemp` file and curl gets `@file`; the regression test + is the honest shape — real listener, real `curl`, a 300 KB fixture pack — + so it fails with the same kernel error the host produced. Every shipped + pack sits well over the ~96 KB cliff, so no small fixture could ever have + seen this. +- **A trial law did not survive deployment.** Against an empty registry the + first synthesis of a freshly registered voice cost ~1.9 s, the next ~1.2 s. + The trial's 6.2–8.0 s cold / 0.5 ms restore ladder, which the warmer's own + header repeated, is retired: warming buys registration — mandatory, because + the registry is memory-only — plus about **0.6 s**. Corrected in + [[Frank TTS Engine]] law 1 and its troubleshooting table, and in + [[Local TTS and Voice Cloning]] and [[Voice Packs]]. +- Sweep afterwards: the retired ladder still lived in six places the first + pass missed — both quadlet files, `scripts/frank-tts-engine.env`, + `scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row, the + research brief's consequences paragraph, and one test comment. All + re-keyed to the measured claim. The trial entry earlier in this log is + left as written — a dated worklog entry is a record — and this line is the + correction that points at it. ## 2026-10-18 — 7772 is live and the harness speaks through it - - Merged **#15** (`0315a62`, the warmer's argument-cap fix) and **#16** - (`d2ce669`, the comment sweep), then deployed both voice services in order: - `--service frank-tts-engine` (image rebuilt from the fixed warmer, unit - `active`, zero drop-ins, warm pass in the unit's own start — `narrator1 - cold=1542ms warm=1709ms`, `narrator2 1329/1405`, `narrator3 1451/1394`, `3 - voice(s) registered and warm`) and `--service frank-tts`. The Playwright gate - ran at the end of each deploy: 33/33 twice. -- Deployed-state proof from a LAN client: `/health` → `omni-http` with all three - voices; the same sentence in `narrator1` and `narrator2` came back with - **different sha256** and different RMS (4.24 s, 24 kHz mono PCM16), so the name - actually selects a voice. Engine-side ledger for the harness request: - `TTFA 67.5 ms`, `Total 2158.8 ms`, `audio 3.76 s, RTF 0.574`. + (`d2ce669`, the comment sweep), then deployed both voice services in order + (`--service frank-tts-engine`, then `--service frank-tts`); Playwright gate + 33/33 twice. Deployed-state proof from a LAN client: `/health` → + `omni-http` with all three voices, and the same sentence in `narrator1` vs + `narrator2` came back with different sha256 and different RMS — the name + actually selects a voice. - **The harness `speech` role moved off Kokoro**: `config.yml` - `modelRoles.speech: frank-tts (dankclaw)/frank-tts`. No voice key moves with it. - The bundle is unambiguous: the speech device forwards the call's `voice_id` and - nothing else, `tts.localVoice` belongs to the local backend (it is what answered - in the control test), and `speech.voice` is the TUI's speak-aloud enum. -- Two harness laws learned by breaking it first (both now on [[Frank TTS]]): an - explicit speech role has **no fallback candidate**, so a dead endpoint errors - instead of degrading; and `models.yml` loads at process start while - `config.yml` is read live, so a session older than the `frank-tts` row fails - exactly like a dead endpoint. Fresh-process proof in the same minute: - `Saved 180524 bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1)` - with the matching façade log line `speech voice=narrator1 format=wav chars=51 - -> 180524 bytes`. This window's own session is that stale case and knows it. -- Status sweep, because six pages still said the endpoint did not exist: both - vault descriptions, the two `index.md` bullets, the Open Notebook blocker (the - voice leg works; open-notebook itself is what is missing), the engine page's - "Deploy order — nothing below has been run" is now a **Deploy record**, and the - homepage card lost `wip: down` so the admin panel probes 7772 for real. -- Corrected the same evening, by breaking it in a restarted session: a harness - call that names no voice reaches 7772 with **no `voice` key at all** — `400 - unknown voice None; available voices: narrator1, narrator2, narrator3` — - because no setting feeds one. The earlier `Saved 180524 bytes - (voice=narrator1)` succeeded only because that run was *told* to pass - `voice_id`. The endpoint therefore grew `DEFAULT_VOICE` (its own PR): a - voice-less request speaks the declared pack and the journal line carries - `(default)`, while an unknown name still 400s with the list. Reverting the role - to Kokoro is one key, not two, and `speech.voice: narrator1` — briefly recorded - above as wiring — went back to `am_fenrir`: that key is the vocalizer's Kokoro - enum, so a pack name there breaks speak-aloud and does nothing for synthesis. + `modelRoles.speech: frank-tts (dankclaw)/frank-tts`. No voice key moves + with it. +- Two harness laws learned by breaking it first (both now on [[Frank TTS]]): + an explicit speech role has **no fallback candidate**, so a dead endpoint + errors instead of degrading; and `models.yml` loads at process start while + `config.yml` is read live, so a session older than the `frank-tts` row + fails exactly like a dead endpoint. +- Corrected the same evening, by breaking it in a restarted session: a + harness call that names no voice reaches 7772 with **no `voice` key at + all** — `400 unknown voice None; available voices: …` — because no setting + feeds one. The endpoint therefore grew `DEFAULT_VOICE` (its own PR): a + voice-less request speaks the declared pack, an unknown name still 400s + with the list. `speech.voice: narrator1` — briefly recorded as wiring — + went back to `am_fenrir`: that key is the vocalizer's Kokoro enum, so a + pack name there breaks speak-aloud and does nothing for synthesis. +- Status sweep: six pages still said the endpoint did not exist — both vault + descriptions, the two `index.md` bullets, the Open Notebook blocker (the + voice leg works; open-notebook itself is what is missing), the engine + page's deploy order (now a deploy record), and the homepage card, which + lost `wip: down` so the admin panel probes 7772 for real. ## 2026-10-18 — the default voice shipped: a harness voiceover needs no voice - -- Merged **#17** (`b590695`) and **#18** (`7889a68`), then deployed both affected - services in one dispatch — `--service frank-tts --service homepage`, 159 s, - Playwright gate 33/33. The shim image rebuilt with the new handler; the - homepage checker took the card on its next timer tick: `frank-tts → up`, - probed for real rather than gated. -- The call that had failed is the one that proves the fix. Same device, same - restarted session, **no** `voice_id`: `Saved 430124 bytes (model=frank-tts - (dankclaw)/frank-tts, voice=default, codec=wav)`. The façade journal says what - actually spoke — `speech voice=narrator1 (default) format=wav chars=148 -> - 430124 bytes` — with the engine's ledger behind it (`audio 8.96 s, RTF 0.551`). - The harness prints `voice=default` because it never had a name to print; the - voice that answered is the operator's line in `scripts/frank-tts.env`, and the - journal keeps them distinguishable forever after. +- Merged **#17** (`b590695`) and **#18** (`7889a68`) and deployed + `--service frank-tts --service homepage` in one dispatch, Playwright gate + 33/33. The call that had failed is the one that proves the fix: same + device, **no** `voice_id` — `Saved 430124 bytes (…, voice=default, + codec=wav)` — and the façade journal says what actually spoke, + `speech voice=narrator1 (default)`. The harness prints `voice=default` + because it never had a name to print; the voice that answered is the + operator's `DEFAULT_VOICE` line in `scripts/frank-tts.env`, and the journal + keeps them distinguishable forever after. - The page gained a **Default voice** row in its contract table and [[#Naming a voice]] now states that a call naming nothing is the harness's normal shape, not a client bug — the misconception that produced the 400 in the first place. ## 2026-10-18 — OpenWebUI "voice narrator1 not found": the rejector is in the browser - - Frank's toast — `Error: Voice "narrator1" not found. Should be one of: af_heart, …, bm_fable` — looked like frank-tts refusing the name. It is not: that 29-name enum is the voice pack of **Kokoro.js running in the - page**, and OpenWebUI 0.11.4 ships that implementation in its own front - end (frozen table `{af_heart:{name:"Heart",…}}`, `_validate_voice`, model - `onnx-community/Kokoro-82M-v1.0-ONNX`, a personal *Kokoro.js dtype* - setting, engine values `""`/`browser-kokoro`). + page** — OpenWebUI 0.11.4 ships that implementation in its own front end. - Instruments that separated the two paths, in order: the admin config DB is - correct (`audio.tts.engine=openai`, - `audio.tts.openai.api_base_url=http://host.containers.internal:7772/v1`, - `model=frank-tts`, `voice=narrator1`, `split_on=punctuation`); from inside - the container the exact OpenWebUI body - (`{"model":"frank-tts","input":…,"voice":"narrator1","response_format":"mp3","speed":1.0}`) - returns **200 `audio/mpeg` 7,629 B**; and `podman logs open-webui` holds - **zero** `audio/speech` requests for the failing click. A server-path - failure leaves a log line and a frank-tts error — a `browser-kokoro` - failure leaves neither. No container env carries an `AUDIO_*`/`TTS` - variable and no `user.settings` row (all four users) carries an audio - override, so the engine choice is per-browser storage, not fleet state. + correct (`audio.tts.engine=openai`, base `http://host.containers.internal:7772/v1`, + `voice=narrator1`); from inside the container the exact OpenWebUI body + returns **200 `audio/mpeg`**; and `podman logs open-webui` holds **zero** + `audio/speech` requests for the failing click. A server-path failure leaves + a log line and a frank-tts error — a `browser-kokoro` failure leaves + neither — and no container env or `user.settings` row carries an override, + so the engine choice is per-browser storage, not fleet state. - Fix is a UI click, not a deploy: Settings → Audio → TTS Engine → **Default** hands synthesis back to frank-tts and the cloned voices; choosing `af_heart` keeps in-browser Kokoro, which cannot clone (fixed - pack). Recorded on [[OpenWebUI]] under *Text-to-speech (frank-tts)*; - suite green 384/384 (wall 65.9 s). + pack). Recorded on [[OpenWebUI]] under *Text-to-speech (frank-tts)*. - Also in the window: **PR #22 merged** (`0c7fe8b`, `dank/command-center`) so `agent/extensions/dankvoice.ts` landed on the auto-load path — proven in a - plain `omp` session with no flags: `/say merged build check` → 1.20 s WAV - to the player, temp dir cleaned; the audible pass through the default - `pw-play` detection confirmed by Frank. Sessions open before the merge need - a restart to pick the command up. + plain `omp` session with no flags: `/say merged build check` → WAV to the + player, temp dir cleaned. Sessions open before the merge need a restart to + pick the command up. ## 2026-10-18 — frank-zealot: 2 × 262,144 over unquantized f16 KV - -- Frank's instruction: **"shift frank-zealot to 2 parallels of unquantized - kv"**. The env delta is four lines: `CTX_SIZE=524288`, `PARALLEL=2`, - `CACHE_TYPE_K=f16`, `CACHE_TYPE_V=f16`. Everything else holds — batch - **4096** / ubatch **2048**, `--flash-attn on`, `--jinja`, `--metrics`, - `--presence-penalty 1.0`, `SPEC_TYPE=none`, the Q8_0 mmproj, port 7771, and - the per-slot window **262,144** — `models.yml` `contextWindow` untouched. -- **Why the pool had to halve.** Exact KV bills **32,768 B/token** (8 - full-attention layers × 4 KV heads × 256 dim × 2 B × 2 caches), so the old - 786,432 pool asks **24,576 MiB** against the **24,123 MiB** the driver leaves - addressable — impossible, not tight. 524,288 positions ⇒ **16,384 MiB** and - exactly two windows of 262,144. -- Arithmetic correction to the repo's long-standing figure: "the f16 pair would - need **26,112 MiB**" doubled the q8_0 number *including* its 1/32 scale - overhead — q8_0 is 17,408 B/token (1.0625 B/element), so f16/q8_0 is +- 2026-10-18 — frank-zealot: shape cut to 2 × 262,144 over unquantized f16 KV + (the 786,432 pool cannot fit exact KV on the 3090; concurrency 3 → 2, and + MTP is now doubly out — off by the 13:16 measurement, and exact KV bills a + draft-context f16 pool over the whole context). See [[frank-zealot]]. +- Arithmetic correction to the repo's long-standing figure: "the f16 pair + would need **26,112 MiB**" doubled the q8_0 number *including* its 1/32 + scale overhead — q8_0 is 17,408 B/token (1.0625 B/element), so f16/q8_0 is **1.882×**, not 2×, and exact f16 over 786,432 is **24,576 MiB**. The conclusion stands (three f16 windows cannot boot this card); the number was wrong. -- The cost: **concurrency 3 → 2 conversations**, and MTP is now **doubly** out. - The 13:16 measurement still says off, and exact KV adds a second reason — the - draft context bills its own f16 KV over the WHOLE pool (**4,096 B/position** - × 524,288 = **2,048 MiB**) on a trunk projecting ~21,200 used. -- **Projection, explicitly not a measurement:** ~**21,200 MiB** used / - ~**2,920 free**, derived from the 13:19 ledger's 17,922 used because only two - terms move — the KV pool (**+3,328**) and one recurrent-state cell - (**−50.25**, 3 cells → 2). The measured boot ledger and the decode/prefill - re-measure land in the follow-up commit, after the first f16 boot. - New gate in `tests/frank_zealot_unit.bats`: dtype × pool plus the measured - non-KV terms must fit **24,123 MiB** with **≥ 1 GiB** of slack — and the same - gate proves it refuses the impossible arm (**f16 at 786,432**), not merely - passes the shipped one. Runbook re-keyed to the new shape: - `LLM/frank-zealot-llama-prism-bonsai9-mtp.md`. + non-KV terms must fit the driver-addressable card with **≥ 1 GiB** of slack + — and the same gate proves it refuses the impossible arm (f16 at 786,432), + not merely passes the shipped one. diff --git a/tests/docs_completeness.bats b/tests/docs_completeness.bats index 775e3d9..1c16919 100644 --- a/tests/docs_completeness.bats +++ b/tests/docs_completeness.bats @@ -3,6 +3,9 @@ # Every service defined in a compose.yaml must have a matching documentation # page in serverAppConfigs-docs/ and must be reachable from the index. This is the "docs first" # contract of this repo: no service ships without documentation. +# The vault's second law, added 2026-10-18: ONE page per live LLM endpoint. +# A payload swap is an Eras row on that endpoint's page, never a page of its +# own -- a second file holding the same shape is the next thing to go stale. load helpers @@ -43,3 +46,31 @@ load helpers run "$REPO_ROOT/scripts/check-vault-links.py" "$REPO_ROOT/serverAppConfigs-docs" [ "$status" -eq 0 ] || { echo "$output"; false; } } + +@test "one page per live LLM endpoint" { + # The live set is derived, not hardcoded: an endpoint is live if scripts/ + # holds both its env and its unit (frank-tts has an env but no unit, so it + # is not an inference endpoint; homepage-status is the reverse). + local vault="$REPO_ROOT/serverAppConfigs-docs" derived=0 name extra + for envf in "$REPO_ROOT"/scripts/frank-*.env; do + name=$(basename "$envf" .env) + [ -f "$REPO_ROOT/scripts/$name.service" ] || continue + derived=$((derived + 1)) + [ -f "$vault/LLM/$name.md" ] || { + echo "live endpoint '$name' has no canonical page: serverAppConfigs-docs/LLM/$name.md" + return 1 + } + extra=$(find "$vault" -name "${name}-*.md" -not -path '*/.AGENTTRASH/*' | sort) + if [ -n "$extra" ]; then + echo "endpoint '$name' owns exactly one page; these files re-create a second owner of its shape:" + echo "$extra" + echo "Fold them into LLM/$name.md (an Eras row plus, if needed, a Law) and trash the page." + return 1 + fi + done + # Guard against the derivation going vacuous -- an empty loop passes silently. + [ "$derived" -ge 3 ] || { + echo "expected at least 3 live endpoints derived from scripts/*.env + scripts/*.service, got $derived" + return 1 + } +}