comments: replace law idiom with the mechanism it encoded

Every comment that cited a named law, an incident date, or an internal nickname now states the underlying mechanism and why it matters, generically: curl exit-0 artifacts and byte-count gates, argv-in-/proc secret exposure, tag mutability vs digest pinning, shape-specialized kernels refusing uncompiled slot counts, MTP heads as file tensors no flag can supply, drafter/slot coupling as an engine property, pkill -f matching its invoker, pinned-RAM pairing with LimitMEMLOCK, one-encoding-per-table, the short-run first-token noise floor, and the protected-expert rule. No behavior change: comment/string text only; suite green (371 tests).
docs(operations): LLM usage-attribution research catalog
2026-10-14 13:25:41 -07:00 · 2026-10-14 13:25:41 -07:00
455 changed files with 31698 additions and 37667 deletions
@@ -0,0 +1,71 @@
+# Tunables for frank-default-small (Tiel-Coder-35B-A3B UD-Q6_K_XL,
+# qwen35moe hybrid MoE, Dankserver, RTX 3090).
+#
+# Runtime: upstream llama.cpp (latest master, CUDA build with
+# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86) on RTX 3090
+# 24 GB; Ryzen 7 7700X; 62 GB RAM.  Replaced the ik_llama.cpp Ornith
+# CRACK Q8_0 deployment on 2026-09-01.
+#
+# Model: peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF, UD-Q6_K_XL quant
+# (30 GB).  Architecture: qwen35moe — 40 layers, full_attention_interval=4
+# (10 attention + 30 SSM/Mamba), 256 experts / 8 used, 2 KV heads × 256
+# dim, embedding 2048, rope dimension_count 64, rope freq_base 10 000 000.
+# Text-only (no mmproj).  Built-in chat template in the GGUF.
+#
+# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
+# Non-expert weights (~1.6 GB) fit entirely on the 24 GB card.
+#
+# KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per slot
+# × 3 parallel) and 10 attention layers, the KV cache is ~2 GB — the
+# requested 2 GB VRAM buffer.
+#
+# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
+# ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
+# incident). This box is NVIDIA/CUDA, but the rule keeps the env portable.
+
+# Model file (UD-Q6_K_XL, 30 GB, text-only MoE hybrid).
+MODEL_DIR=/home/dank/models/tiel-coder-35b
+MODEL_FILE=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
+
+# Upstream llama.cpp llama-server (CUDA build, latest master).
+# Built with -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86
+# using the staged CUDA 13.2 toolkit at /home/dank/cuda13-stage.
+LLAMA_SERVER=/home/dank/llama.cpp/build/bin/llama-server
+
+ALIAS=frank-default-small
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770
+
+# Total context budget across all parallel slots.  200700 / 3 = 66 900
+# per slot.  The model's context_length is 262 144, so 66 900 is well
+# within range.
+CTX_SIZE=200700
+PARALLEL=3
+
+# Expert offload: all MoE expert weights pinned to CPU RAM.
+CPU_MOE=1
+
+# All non-expert layers on GPU (auto-fit will determine the exact count;
+# with --cpu-moe the non-expert footprint is ~1.6 GB, leaving ample room).
+N_GPU_LAYERS=999
+
+# KV cache: Q8_0 for both K and V.  ~2 GB total for 200700 ctx × 3 slots
+# × 10 attention layers × (2 KV heads × 256 dim × 1 byte) × 2 (K+V).
+CACHE_TYPE_K=q8_0
+CACHE_TYPE_V=q8_0
+
+VERBOSITY=2
+
+# Prefill batch sizing.
+BATCH_SIZE=4096
+UBATCH_SIZE=1024
+
+# Sampling.
+TEMPERATURE=1.0
+TOP_K=20
+TOP_P=0.95
+MIN_P=0.0
+
+# Compute threads lowered so other processes keep a core.
+THREADS=7
+THREADS_HTTP=8
@@ -0,0 +1,16 @@
+[Unit]
+Description=llama.cpp Tiel-Coder-35B-A3B Q6_K_XL qwen35moe hybrid MoE (dankserver, frank-default-small)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-small.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,126 @@
+# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
+# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
+# Strix Halo gfx1151, unified memory).
+#
+# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
+# qwen4exp-on-rocmfpx-d3ca537 patch applied — the model card's pinned
+# runtime: ROCmFP4 tensor types AND the qwen4exp architecture in one
+# tree; stock llama.cpp and charlie's fork alone cannot load this GGUF).
+# Built per the card recipe:
+#   cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
+# at /home/dank/GitPackages/kingjones-ROCmFPX/build.
+# Replaced the agentionai ROCmFP4-FAST imatrix deployment (charlie's
+# build-qwen38 runtime) on 2026-08-30.
+#
+# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
+# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
+# at layer 1), Q4_0_ROCMFP4_STRIX (file_type 105), 113.47 GiB single
+# file. STRIX is the top quality tier: PLE table at Q8_0 (50.66 GiB),
+# token embeddings + lm head at Q6_K. The PLE table NEVER enters GPU
+# memory — it streams from the file through the OS page cache. Two hard
+# card constraints:
+#  1. mmap MUST stay enabled (default) — --no-mmap forces the 50.66 GiB
+#     table into anonymous memory and the cgroup killer takes the
+#     process with nothing in the server log (only dmesg).
+#  2. Do NOT pin the PLE tensor to CPU with -ot — measured gen fell
+#     ~23 -> 13.4 tok/s; the kernel streams it better.
+# NOTE: the card's own build_attn_qsa still calls ggml_fill on the
+# F16-cast kq mask (the path that aborted our charlie build with
+# --flash-attn on before scripts/patches/qwen4exp-qsa-f32-mask.patch).
+# If a clean kingjones-tree build aborts graph build with
+# GGML_ASSERT(a-type == GGML_TYPE_F32), port that fix (F32 base mask +
+# cast at the attention call) into this tree and rebuild.
+#
+# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
+# incident). llama.cpp reads --host/--port/--temp as CLI flags, never as
+# env vars.
+
+# kingjones30/ROCmFPX runtime binary (card recipe build).
+LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
+
+# Model package from
+# https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
+# Layout: single unsharded GGUF, text-only (the repo ships no mmproj;
+# the vision tower below is borrowed from the agentionai FAST package —
+# same base model, f16 — kept only if it loads clean, drop
+# MMPROJ_PATH if it does not).
+MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
+MODEL_FILE=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
+MMPROJ_PATH=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf
+
+# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
+# Sharp template drives thinking on by default (medium effort); the server
+# emits reasoning in DeepSeek format so clients receive reasoning_content
+# separately instead of literal thinking tags in content.
+CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
+CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
+
+# Alias + network.
+ALIAS=frank-dragoon
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770
+
+# Serving: 401408-token context, TWO slots (parallel 2), batch 4096 /
+# ubatch 512, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
+# prompt cache. --fit off: auto-fit segfaults on this model's
+# hyper-connection/indexer config; -ngl 999 puts every layer on the
+# SVM-backed GPU pool. ubatch 512 matches the card's re-measurement
+# (-ub 1024/2048/4096 showed no effect; -t 32 vs 16 is noise).
+#
+# CTX_SIZE=401408 / PARALLEL=2: carried over from the FAST deployment
+# where it loaded clean (~74 s, health 200). The card measures the full
+# 262144 window at ~71.7 GiB GTT at load on the same 128 GB board; QSA
+# caps KV growth so the extra window is cheap. KV footprint scales
+# with the 12 attention layers only. Verify load on first STRIX boot.
+CTX_SIZE=401408
+PARALLEL=2
+
+BATCH_SIZE=4096
+UBATCH_SIZE=512
+NUM_THREADS=16
+THREAD_BATCH=16
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+CTX_CHECKPOINTS=16
+CACHE_RAM=8192
+SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
+SPLIT_MODE=none
+FIT=off
+NGL=999
+FLASH_ATTN=on
+
+# Sampling (card instruct table, non-thinking).
+TEMPERATURE=0.7
+TOP_K=20
+TOP_P=0.8
+MIN_P=0.0
+PRESENCE_PENALTY=1.5
+
+# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
+# keep the reasoning trail in the conversation). DeepSeek format projects the
+# reasoning into the API's reasoning_content field instead of literal thinking
+# tags.
+REASONING=on
+REASONING_FORMAT=deepseek
+
+# Strix Halo unified memory + native gfx1151.
+HSA_OVERRIDE_GFX_VERSION=11.5.1
+
+VERBOSITY=4
+
+
+# Speculative decoding: disabled. The card's Usage block ships no spec
+# flags. The Flash-Next MTP heads do not load on this loader (requires
+# per-layer blk.N.hc_attn_norm; every head carries only blk.48 —
+# verified 3x on build-qwen38). DRAFT_MODEL_* point at the retired
+# agentionai FAST head and are unused while SPEC_TYPE=none.
+DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
+DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
+SPEC_TYPE=none
+SPEC_DRAFT_N_MAX=3
+SPEC_DRAFT_N_MIN=0
+SPEC_DRAFT_P_MIN=0.75
+SPEC_DRAFT_NGL=999
+SPEC_DRAFT_DEVICE=ROCm0
+SPEC_DRAFT_TYPE_K=f16
+SPEC_DRAFT_TYPE_V=f16
@@ -0,0 +1,16 @@
+[Unit]
+Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,23 @@
+# Tunables for frank-npu (FastFlowLM NPU endpoint, Dankclaw)
+# Runtime: FLM 1.0.1 (/usr/bin/flm) on the XDNA2 NPU (50 TOPS). Boot tag is
+# nuextract3:4b (NuExtract3-4B-NPU2, Qwen3.5-4B skeleton, structured-extraction
+# model); serve routes every request by the exact FLM registry tag in the
+# request's model field, so any registered tag (gemma4-it:e4b, qwythos:9b, ...)
+# is reachable on this port regardless of the boot tag. The NPU is a
+# single-lane device: generations serialize FIFO, and the first request after a
+# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
+# pool shared with frank-dragoon (iGPU).
+FLM_MODEL=nuextract3:4b
+FLM_HOST=0.0.0.0
+FLM_PORT=7771
+# turbo = max NPU power mode (xrt-smi configure --pmode turbo at serve start).
+# No other tenant uses the NPU on this host, so the top mode costs nothing to
+# neighbours; drop to `performance` if iGPU power-budget contention ever shows.
+FLM_PMODE=turbo
+# FLM_ASR=1 additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes
+# /v1/audio/transcriptions. NuExtract3 answers text-only; audio stays off.
+FLM_ASR=0
+# -1 = no --ctx-len flag: every pack loads with its own trained context
+# (nuextract3:4b -> 32768, gemma4-it:e4b -> 131072). A global override would
+# over-allocate KV for the small packs.
+FLM_CTX_LEN=-1
@@ -0,0 +1,17 @@
+[Unit]
+Description=FastFlowLM NPU sight+sound inference (frank-npu) on Dankclaw
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+Environment="FRANK_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-npu.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-npu.sh
+LimitMEMLOCK=infinity
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,180 @@
+#!/usr/bin/env bash
+# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
+# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
+# Dankclaw (Strix Halo gfx1151, unified memory).
+#
+# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
+# model card's qwen4exp-on-rocmfpx patch) built at
+# /home/dank/GitPackages/kingjones-ROCmFPX/build (HIP, gfx1151,
+# ROCm 7.2.4, card recipe). The ROCmFP4 quant types plus the qwen4exp
+# architecture must be in one tree; stock llama.cpp cannot load this
+# GGUF.
+#
+# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
+# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
+# at layer 1) quantized Q4_0_ROCMFP4_STRIX, 113.47 GiB single file. The
+# 50.66 GiB PLE n-gram table stays OUT of GPU memory and streams via
+# mmap from the OS page cache — never add --no-mmap or -ot for it.
+#
+# Profile: card sampling table (temp 0.7, top_p 0.80, top_k 20, min_p 0.0,
+# presence 1.5 — non-thinking instruct) mixed with the validated serving
+# profile: ctx 401408 x 2 slots, b 4096 / ub 512, t 16 / tb 16, f16 KV,
+# fa on (if the graph build aborts on the QSA F16 mask, see the note in
+# the env file: port scripts/patches/qwen4exp-qsa-f32-mask.patch into
+# this tree and rebuild), cont-batching, prompt cache + 8 GiB cache-ram
+# + idle-slot cache, 16 ctx checkpoints, --fit off (auto-fit segfaults
+# on qwen4exp hyper-connection config). Speculative decoding off: the
+# Flash-Next MTP heads do not load on this loader; the card ships no
+# spec flags.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
+if [[ -f "$ENV_FILE" ]]; then
+    set -a
+    # shellcheck disable=SC1090
+    source "$ENV_FILE"
+    set +a
+fi
+
+# --- defaults (used when the env file does not set them) -------------------
+: "${LLAMA_BIN:=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
+: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
+: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
+: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
+: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
+: "${CHAT_TEMPLATE_KWARGS:={}}"
+: "${ALIAS:=frank-dragoon}"
+: "${BIND_ADDR:=0.0.0.0}"
+: "${LISTEN_PORT:=7770}"
+: "${CTX_SIZE:=401408}"
+: "${PARALLEL:=2}"
+: "${BATCH_SIZE:=4096}"
+: "${UBATCH_SIZE:=512}"
+: "${NUM_THREADS:=16}"
+: "${THREAD_BATCH:=16}"
+: "${CACHE_TYPE_K:=f16}"
+: "${CACHE_TYPE_V:=f16}"
+: "${CTX_CHECKPOINTS:=16}"
+: "${CACHE_RAM:=8192}"
+: "${SLOT_SAVE_PATH:=$SCRIPT_DIR/slot-state}"
+: "${SPLIT_MODE:=none}"
+: "${FIT:=off}"
+: "${NGL:=999}"
+: "${FLASH_ATTN:=on}"
+: "${TEMPERATURE:=0.7}"
+: "${TOP_K:=20}"
+: "${TOP_P:=0.8}"
+: "${MIN_P:=0.0}"
+: "${PRESENCE_PENALTY:=1.5}"
+: "${REASONING:=on}"
+: "${REASONING_FORMAT:=deepseek}"
+: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
+: "${SPEC_TYPE:=none}"
+: "${DRAFT_MODEL_DIR:=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST}"
+: "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
+: "${SPEC_DRAFT_N_MAX:=3}"
+: "${SPEC_DRAFT_N_MIN:=0}"
+: "${SPEC_DRAFT_P_MIN:=0.75}"
+: "${SPEC_DRAFT_NGL:=999}"
+: "${SPEC_DRAFT_DEVICE:=ROCm0}"
+: "${SPEC_DRAFT_TYPE_K:=f16}"
+: "${SPEC_DRAFT_TYPE_V:=f16}"
+
+MODEL="$MODEL_DIR/$MODEL_FILE"
+for f in "$LLAMA_BIN" "$MODEL" "$CHAT_TEMPLATE_FILE"; do
+    if [[ ! -r "$f" ]]; then
+        echo "missing or unreadable required artifact: $f" >&2
+        exit 1
+    fi
+done
+if [[ ! -x "$LLAMA_BIN" ]]; then
+    echo "missing executable runtime: $LLAMA_BIN" >&2
+    exit 1
+fi
+
+# --- speculative decoding (MTP draft head) ---------------------------------
+# The MTP draft head is a separate GGUF sidecar loaded via --spec-draft-model
+# (alias --model-draft).  ngram-only specs do not need a draft file.
+# Set SPEC_TYPE=none in the env to disable speculative decoding entirely.
+SPEC_ARGS=()
+if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
+    SPEC_ARGS=(
+        --spec-type "$SPEC_TYPE"
+        --spec-draft-n-max "$SPEC_DRAFT_N_MAX"
+        --spec-draft-n-min "$SPEC_DRAFT_N_MIN"
+        --spec-draft-p-min "$SPEC_DRAFT_P_MIN"
+    )
+    if [[ "$SPEC_TYPE" == *draft-mtp* ]]; then
+        DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
+        if [[ ! -r "$DRAFT_MODEL" ]]; then
+            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
+            exit 1
+        fi
+        SPEC_ARGS+=(
+            --spec-draft-model "$DRAFT_MODEL"
+            --spec-draft-ngl "$SPEC_DRAFT_NGL"
+            --spec-draft-device "$SPEC_DRAFT_DEVICE"
+            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
+            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
+        )
+    fi
+    if [[ -n "${SPEC_DRAFT_P_SPLIT:-}" ]]; then
+        SPEC_ARGS+=(--spec-draft-p-split "$SPEC_DRAFT_P_SPLIT")
+    fi
+fi
+
+# The STRIX repo ships no mmproj; this is the agentionai FAST package's
+# f16 vision tower for the same base model. Passed only when readable.
+MMPROJ_ARGS=()
+if [[ -n "${MMPROJ_PATH:-}" && -r "${MMPROJ_PATH:-}" ]]; then
+    MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
+fi
+
+SERVER_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+mkdir -p "$SLOT_SAVE_PATH"
+ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
+LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+
+exec /usr/bin/env \
+    ROCM_PATH="$ROCM_PATH" \
+    HIP_VISIBLE_DEVICES=0 \
+    LD_LIBRARY_PATH="$LIBRARY_PATH" \
+    HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
+    "$LLAMA_BIN" \
+    --model "$MODEL" \
+    --alias "$ALIAS" \
+    --chat-template-file "$CHAT_TEMPLATE_FILE" \
+    --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS" \
+    --host "$BIND_ADDR" \
+    --port "$LISTEN_PORT" \
+    --jinja \
+    --device ROCm0 \
+    --gpu-layers "$NGL" \
+    --split-mode "$SPLIT_MODE" \
+    --parallel "$PARALLEL" \
+    --ctx-size "$CTX_SIZE" \
+    --batch-size "$BATCH_SIZE" \
+    --ubatch-size "$UBATCH_SIZE" \
+    --fit "$FIT" \
+    --flash-attn "$FLASH_ATTN" \
+    --cache-type-k "$CACHE_TYPE_K" \
+    --cache-type-v "$CACHE_TYPE_V" \
+    --threads "$NUM_THREADS" \
+    --threads-batch "$THREAD_BATCH" \
+    --ctx-checkpoints "$CTX_CHECKPOINTS" \
+    --cache-ram "$CACHE_RAM" \
+    --slot-save-path "$SLOT_SAVE_PATH" \
+    --cache-prompt \
+    --cache-idle-slots \
+    --metrics \
+    --temp "$TEMPERATURE" \
+    --top-p "$TOP_P" \
+    --top-k "$TOP_K" \
+    --min-p "$MIN_P" \
+    --presence-penalty "$PRESENCE_PENALTY" \
+    --reasoning "$REASONING" \
+    --reasoning-format "$REASONING_FORMAT" \
+    --reasoning-budget -1 \
+    "${MMPROJ_ARGS[@]}" \
+    "${SPEC_ARGS[@]}"
@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+# Serve the FastFlowLM (FLM) sight+sound endpoint — frank-npu.
+#
+# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
+# single-lane device: generations serialize in FIFO order, so this is a
+# background/triage endpoint, not an interactive multi-user chat. The NPU
+# draws from the unified RAM pool shared with frank-dragoon (the iGPU
+# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
+# for the memory budget and the model-selection tradeoff.
+#
+# NOTE: FLM requires unlimited memlock. The unit sets LimitMEMLOCK=infinity.
+# NOTE: an unknown model tag does NOT 404 — /v1/chat/completions answers HTTP 200 with a
+# flat {"error":"Failed to load <tag> model!"} body and no choices; /v1/completions returns a
+# nested OpenAI-shaped server_error (code 500) also under HTTP 200. Client model IDs must be the exact FLM registry
+# tag (e.g. gemma4-it:e4b), never a service alias. Custom tags only resolve
+# when FLM_CONFIG_PATH points at the user registry that carries them and their
+# kernels are symlinked into /home/dank/.config/flm/xclbins/<PackName>.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_NPU_ENV_FILE:-$SCRIPT_DIR/frank-npu.env}"
+if [[ -f "$ENV_FILE" ]]; then
+    set -a
+    # shellcheck disable=SC1090
+    source "$ENV_FILE"
+    set +a
+fi
+
+# --- defaults (used when the env file does not set them) -------------------
+: "${FLM_BIN:=/usr/bin/flm}"
+: "${FLM_MODEL:=gemma4-it:e4b}"
+: "${FLM_HOST:=0.0.0.0}"
+: "${FLM_PORT:=7771}"
+: "${FLM_PMODE:=turbo}"
+: "${FLM_ASR:=0}"
+: "${FLM_CTX_LEN:=131072}"
+
+# The user registry is a superset of the system registry (it carries the
+# custom tags too), and its model_path resolves relative to the registry file,
+# so exports keep a tag swap in the env file from needing script edits.
+export FLM_CONFIG_PATH="${FLM_CONFIG_PATH:-/home/dank/.config/flm/model_list.json}"
+export FLM_XCLBIN_PATH="${FLM_XCLBIN_PATH:-/home/dank/.config/flm/xclbins}"
+
+if [[ ! -x "$FLM_BIN" ]]; then
+    echo "missing flm binary: $FLM_BIN" >&2
+    exit 1
+fi
+
+# --- optional flags ----------------------------------------------------------
+OPT_ARGS=()
+if [[ "${FLM_ASR}" != "0" ]]; then
+    OPT_ARGS+=(--asr "${FLM_ASR}")
+fi
+if [[ "${FLM_CTX_LEN}" != "-1" ]]; then
+    OPT_ARGS+=(--ctx-len "${FLM_CTX_LEN}")
+fi
+
+exec "$FLM_BIN" serve "$FLM_MODEL" \
+    --host "$FLM_HOST" \
+    -p "$FLM_PORT" \
+    --pmode "$FLM_PMODE" \
+    "${OPT_ARGS[@]}"
@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Serve peculiar-ragdoll/Tiel-Coder-35B-A3B (UD-Q6_K_XL) as
+# frank-default-small via stock llama.cpp on Dankserver (RTX 3090).
+#
+# Runtime: upstream llama.cpp (latest master, CUDA build with
+# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86).
+# The model is a 35B-total / 8-active MoE hybrid (qwen35moe arch) with
+# SSM/Mamba layers (full_attention_interval=4: 10 attention + 30 SSM
+# layers out of 40).  256 experts, 8 used per token.
+#
+# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
+# Non-expert weights (attention, SSM, shared expert, embeddings, output)
+# are ~1.6 GB and fit entirely on the 24 GB RTX 3090.
+#
+# KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per
+# slot × 3 parallel) and 10 attention layers (2 KV heads × 256 dim),
+# the KV cache is ~2 GB — the requested 2 GB VRAM buffer.
+#
+# Model is loaded with mmap (default).  The 30 GB Q6_K_XL file is
+# memory-mapped; only accessed pages enter RAM.
+#
+# Tunables come from frank-default-small.env unless FRANK_SMALL_ENV_FILE
+# overrides that path.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_SMALL_ENV_FILE:-$SCRIPT_DIR/frank-default-small.env}"
+if [[ -f "$ENV_FILE" ]]; then
+    set -a
+    # shellcheck disable=SC1090
+    source "$ENV_FILE"
+    set +a
+fi
+
+# --- defaults (used when the env file does not set them) -----------------
+: "${MODEL_DIR:=/home/dank/models/tiel-coder-35b}"
+: "${MODEL_FILE:=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf}"
+: "${LLAMA_SERVER:=/home/dank/llama.cpp/build/bin/llama-server}"
+: "${ALIAS:=frank-default-small}"
+: "${BIND_ADDR:=0.0.0.0}"
+: "${LISTEN_PORT:=7770}"
+: "${CTX_SIZE:=200700}"
+: "${PARALLEL:=3}"
+: "${CACHE_TYPE_K:=q8_0}"
+: "${CACHE_TYPE_V:=q8_0}"
+: "${N_GPU_LAYERS:=999}"
+: "${BATCH_SIZE:=4096}"
+: "${UBATCH_SIZE:=1024}"
+: "${VERBOSITY:=2}"
+: "${CPU_MOE:=1}"
+: "${TEMPERATURE:=1.0}"
+: "${TOP_K:=20}"
+: "${TOP_P:=0.95}"
+: "${MIN_P:=0.0}"
+: "${THREADS:=7}"
+: "${THREADS_HTTP:=8}"
+
+MODEL="$MODEL_DIR/$MODEL_FILE"
+
+if [[ ! -f "$MODEL" ]]; then
+    echo "missing required file: $MODEL" >&2
+    exit 1
+fi
+if [[ ! -x "$LLAMA_SERVER" ]]; then
+    echo "missing llama-server binary: $LLAMA_SERVER" >&2
+    exit 1
+fi
+
+exec "$LLAMA_SERVER" \
+    --model "$MODEL" \
+    $([ "$CPU_MOE" = "1" ] && printf -- "--cpu-moe") \
+    --cache-type-k "$CACHE_TYPE_K" \
+    --cache-type-v "$CACHE_TYPE_V" \
+    --ctx-size "$CTX_SIZE" \
+    --parallel "$PARALLEL" \
+    --gpu-layers "$N_GPU_LAYERS" \
+    --batch-size "$BATCH_SIZE" \
+    --ubatch-size "$UBATCH_SIZE" \
+    --temp "$TEMPERATURE" \
+    --top-k "$TOP_K" \
+    --top-p "$TOP_P" \
+    --min-p "$MIN_P" \
+    --threads "$THREADS" \
+    --threads-http "$THREADS_HTTP" \
+    --alias "$ALIAS" \
+    --host "$BIND_ADDR" \
+    --port "$LISTEN_PORT"
@@ -1,16 +1,5 @@
 # Server App Configs

-### AI Disclosures:
-
- Slop rating: 2/5 - Actively using, actively de-slopping, **repository mostly human owned**, lacks formal process.
- Completeness: Unfinished, planning planned, no plan yet
- Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
- Local model: Qwen 3.8 Flash Next
-
-### slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
-### slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
-
-
 The single repository for the two homelab servers: the Podman container
 stacks **and** the LLM inference fleet. One repo, one checkout per host,
 one deploy script.
@@ -79,23 +79,3 @@ MARRETA_APP_KEY=changeme
 # account on first boot from these values.
 MARRETA_ADMIN_EMAIL=admin@franksdank.cloud
 MARRETA_ADMIN_PASSWORD=changeme
-
-# open-notebook (podcast generator; internal-only, no vhost).
-# deploy-remote.sh seeds /home/dank/Containers/open-notebook/env from these
-# values (only if that file does not already exist) and refuses to start the
-# units while any of them is blank: a blank OPEN_NOTEBOOK_PASSWORD means
-# authentication OFF, and a blank encryption key means no provider credential
-# can be saved. The seed writes SURREAL_PASS too — the spelling the `surreal`
-# CLI reads — from SURREAL_PASSWORD, so the app and the database cannot be
-# given different values.
-# OPEN_NOTEBOOK_ENCRYPTION_KEY: encrypts the stored AI provider keys at rest.
-#   Generate: openssl rand -hex 32
-# Changing it after credentials are saved makes them undecryptable.
-OPEN_NOTEBOOK_ENCRYPTION_KEY=changeme
-# UI/API password. Every API route — including the episode-audio download the
-# app serves without a session — is open to the LAN while this is unset.
-OPEN_NOTEBOOK_PASSWORD=changeme
-# SurrealDB root password (the root user NAME comes from
-# scripts/open-notebook.env, so this file holds only the secret).
-#   Generate: openssl rand -base64 32
-SURREAL_PASSWORD=changeme
@@ -162,25 +162,6 @@ marreta.franksdank.cloud {
 	}
 }

-# --- Open Notebook (guarded: Anubis → Authentik; notebook-proxy provider, internal_host caddy:8017) ---
-# The podcast generator. ONE upstream carries the whole app: the image's own
-# Next.js layer rewrites /api/* to the FastAPI beside it (INTERNAL_API_URL,
-# default http://localhost:5055 inside the container), so a browser on this
-# vhost never needs the API port. That is why scripts/open-notebook.env sets
-# API_URL to exactly this origin — left unset, the app auto-detects
-# `<host-you-opened>:5055`, and 5055 on this host belongs to seerr — and why
-# the published API port sits on loopback. App-level login
-# (OPEN_NOTEBOOK_PASSWORD) stays on behind the chain: two owners, edge identity
-# and the app's own session.
-# Dialled by published LAN port, not container name: open-notebook is a
-# rootless quadlet and this Caddy lives in root's storage, so root's DNS does
-# not see the container (the same reason marreta and SearXNG are dialled by IP).
-notebook.franksdank.cloud {
-	reverse_proxy anubis:8923 {
-		header_up X-Real-Ip {remote_host}
-	}
-}
-
 # --- Pi-hole (quadlet on slirp4netns; published 8083) ---
 pihole.franksdank.cloud {
 	basicauth {
@@ -439,16 +420,6 @@ http://:8017 {
 	handle @marreta {
 		reverse_proxy 192.168.0.143:8082
 	}
-
-	@notebook host notebook.franksdank.cloud
-	handle @notebook {
-		# Bare proxy, like ai.franksdank.cloud: the app's SSE answers
-		# (/api/search/ask, chat sessions) stream for minutes through this
-		# same listener with no flush or timeout tuning, so nothing here
-		# needs it — a read/write timeout on the route would be the thing
-		# that cuts them.
-		reverse_proxy 192.168.0.143:8502
-	}
 }

 # --- Optional: Jackett ---
@@ -23,7 +23,7 @@ Network=dankclaw_default
 # Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
 # the tag's nested manifest list, so pin the single-platform digest).
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
-Image=ghcr.io/goauthentik/ldap:2026.8.5@sha256:4c75bd27918418d2d6fbbfe4581bbd5a199fe38942d24559c4483edd64a1ea87
+Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
 Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
 Environment=AUTHENTIK_INSECURE=false
 Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}
@@ -20,7 +20,7 @@ Wants=authentik-server.service

 [Container]
 ContainerName=authentik-outpost
-Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -14,7 +14,7 @@ After=network.target

 [Container]
 ContainerName=authentik-postgres
-Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab
+Image=docker.io/library/postgres:16-alpine
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=POSTGRES_DB=authentik
@@ -19,7 +19,7 @@ Wants=authentik-postgres.service

 [Container]
 ContainerName=authentik-server
-Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=server
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -16,7 +16,7 @@ Wants=authentik-server.service

 [Container]
 ContainerName=authentik-worker
-Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -15,7 +15,7 @@ After=network.target
 [Container]
 ContainerName=gitea-postgres
 Network=gitea.network
-Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab
+Image=docker.io/library/postgres:16-alpine
 EnvironmentFile=/home/dank/Containers/gitea/env
 Environment=POSTGRES_USER=gitea
 Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD}
@@ -19,7 +19,7 @@ After=gitea-postgres.service network.target
 [Container]
 ContainerName=gitea-server
 Network=gitea.network
-Image=docker.io/gitea/gitea:latest@sha256:30777c91280d973fa02be48b1ade735bd6859e8f6f7af3b4444d519437852504
+Image=docker.io/gitea/gitea:latest
 EnvironmentFile=/home/dank/Containers/gitea/env
 Environment=DATABASE_TYPE=postgres
 Environment=DATABASE_HOST=gitea-postgres
@@ -29,13 +29,6 @@ Environment=DATABASE_PASSWD=${GITEA_DB_PASSWORD}
 Environment=DATABASE_PORT=5432
 Environment=USER_UID=1000
 Environment=USER_GID=1000
-# Prometheus scrapes /metrics over the published LAN port (job "gitea" in
-# shared/prometheus/prometheus.yml). Off, /metrics is a 404 and the target is
-# permanently red (the audit's "disabled metrics" class). 0.0.0.0 binds inside
-# the container; the only route in is this host's published 3000, and the
-# exposed surface is counters only — same posture as the LLM /metrics.
-Environment=GITEA__metrics__ENABLED=true
-Environment=GITEA__metrics__ENABLED_HOST=0.0.0.0
 PublishPort=0.0.0.0:3000:3000
 PublishPort=2222:22
 Volume=/home/dank/Containers/gitea/data:/data
@@ -25,7 +25,7 @@ Description=Chrome DevTools Protocol daemon (headless-shell) for hermes
 [Container]
 ContainerName=chrome-devtools-protocol
 Network=host
-Image=docker.io/chromedp/headless-shell@sha256:7bee201478b2f6266aae01243740ee85c16281e32f42d3aed9fdf04f4eace7a1
+Image=docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04
 Entrypoint=/headless-shell/headless-shell
 Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222
 ShmSize=1073741824
@@ -1,43 +0,0 @@
-# If you change this file or any /etc/default/grub.d/*.cfg file,
-# run 'update-grub' afterwards to update /boot/grub/grub.cfg.
-# For full documentation of the options in these files, see:
-#   info -f grub -n 'Simple configuration'
-
-GRUB_DEFAULT=0
-GRUB_TIMEOUT_STYLE=hidden
-GRUB_TIMEOUT=0
-GRUB_DISTRIBUTOR=`( . /etc/os-release && echo ${NAME} )`
-# Identity (passthrough) DMA since 2026-10-18: the IOMMU stays enabled, so the
-# XDNA2 NPU (/dev/accel/accel0, PASID/SVA) is usable again, while the iGPU's DMA
-# runs untranslated. Decided in serverAppConfigs-docs/LLM/
-# dankclaw-iommu-translation-off.md; install with scripts/set-dankclaw-bootline.sh.
-GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=pt"
-
-# If your computer has multiple operating systems installed, then you
-# probably want to run os-prober. However, if your computer is a host
-# for guest OSes installed via LVM or raw disk devices, running
-# os-prober can cause damage to those guest OSes as it mounts
-# filesystems to look for things.
-#GRUB_DISABLE_OS_PROBER=false
-
-# Uncomment to enable BadRAM filtering, modify to suit your needs
-# This works with Linux (no patch required) and with any kernel that obtains
-# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
-#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
-
-# Uncomment to disable graphical terminal
-#GRUB_TERMINAL=console
-
-# The resolution used on graphical terminal
-# note that you can use only modes which your graphic card supports via VBE/GOP/UGA
-# you can see them in real GRUB with the command `videoinfo'
-#GRUB_GFXMODE=640x480
-
-# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
-#GRUB_DISABLE_LINUX_UUID=true
-
-# Uncomment to disable generation of recovery mode menu entries
-#GRUB_DISABLE_RECOVERY="true"
-
-# Uncomment to get a beep at grub start
-#GRUB_INIT_TUNE="480 440 1"
@@ -1,65 +0,0 @@
-#!/bin/sh
-# halogen-fabric-clock hold | release | status: the AMD GPU's fabric clock (FCLK), held at its top level.
-#
-# On Strix Halo the GPU and the NPU working at the same time can hang the machine, and corrupt the NPU's results,
-# while the GPU's fabric clock changes speed; held at its top level, the two run together cleanly. halogen-flash-server
-# with HALOGEN_NPU_MODELS refuses to start until the clock is held. This holds it: the performance level `manual` and
-# only the top pp_dpm_fclk level, on every amdgpu device that has the control (the GPU and SoC clocks stay automatic).
-# Cost: about 1 W at idle. Run as root; halogen-fabric-clock.service runs `hold` at every boot and after a resume.
-#
-#   hold     hold it (waits up to 60 s for the GPU driver to come up)
-#   release  give it back to the driver (the performance level `auto`)
-#   status   print each device's state; exit 0 when every device is held
-set -u
-SYS="${HALOGEN_SYSFS:-/sys}"
-
-devices() {
-  for d in "$SYS"/class/drm/card*/device; do
-    [ -w "$d/pp_dpm_fclk" ] || [ -r "$d/pp_dpm_fclk" ] || continue
-    [ -e "$d/power_dpm_force_performance_level" ] || continue
-    echo "$d"
-  done | sort -u
-}
-
-held() {   # held DEVICE_DIR: the performance level high, or manual with only the top fclk level selected
-  perf=$(cat "$1/power_dpm_force_performance_level" 2>/dev/null)
-  [ "$perf" = high ] && return 0
-  [ "$perf" = manual ] || return 1
-  top=$(grep . "$1/pp_dpm_fclk" | tail -n 1)
-  [ "$(grep -c '\*' "$1/pp_dpm_fclk")" = 1 ] && [ "${top%\*}" != "$top" ]
-}
-
-cmd="${1:-status}"
-case "$cmd" in
-  hold)
-    n=0
-    while [ -z "$(devices)" ] && [ "$n" -lt 60 ]; do sleep 1; n=$((n + 1)); done
-    [ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control (pp_dpm_fclk) under $SYS/class/drm" >&2; exit 1; }
-    rc=0
-    for d in $(devices); do
-      lvl=$(grep . "$d/pp_dpm_fclk" | tail -n 1 | cut -d: -f1)
-      # the driver applies the level several seconds after the write, so the read-back waits up to 30 s
-      ok=0
-      if echo manual > "$d/power_dpm_force_performance_level" && echo "$lvl" > "$d/pp_dpm_fclk"; then
-        t=0; until held "$d" || [ "$t" -ge 300 ]; do sleep 0.1; t=$((t + 1)); done
-        held "$d" && ok=1
-      fi
-      if [ "$ok" = 1 ]; then
-        echo "halogen-fabric-clock: $d: held at level $lvl ($(grep . "$d/pp_dpm_fclk" | tail -n 1 | awk '{print $2}'))"
-      else
-        echo "halogen-fabric-clock: $d: could not hold it (level $lvl)" >&2; rc=1
-      fi
-    done
-    exit "$rc" ;;
-  release)
-    for d in $(devices); do echo auto > "$d/power_dpm_force_performance_level" && echo "halogen-fabric-clock: $d: released (auto)"; done ;;
-  status)
-    [ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control"; exit 1; }
-    rc=0
-    for d in $(devices); do
-      if held "$d"; then echo "$d: held ($(cat "$d/power_dpm_force_performance_level"))"
-      else echo "$d: NOT held ($(cat "$d/power_dpm_force_performance_level"))"; rc=1; fi
-    done
-    exit "$rc" ;;
-  *) echo "usage: halogen-fabric-clock hold | release | status" >&2; exit 2 ;;
-esac
@@ -1,19 +0,0 @@
-# Holds the AMD GPU's fabric clock at its top level at boot and after every resume, for halogen-flash-server with
-# the NPU (HALOGEN_NPU_MODELS): on Strix Halo the GPU and the NPU at once are safe only with that clock held.
-# Install (as root), from this directory:
-#   install -m 755 halogen-fabric-clock /usr/local/sbin/
-#   install -m 644 halogen-fabric-clock.service /etc/systemd/system/
-#   systemctl daemon-reload && systemctl enable --now halogen-fabric-clock.service
-# Check: halogen-fabric-clock status. Undo: systemctl disable halogen-fabric-clock.service, then
-#   halogen-fabric-clock release (or reboot). The unit runs once at boot and once after each resume (a resume can hand
-#   the clock back to the driver); it holds nothing open, so it shows inactive between runs.
-[Unit]
-Description=Hold the AMD GPU fabric clock at its top level (GPU and NPU together)
-After=systemd-modules-load.service suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
-
-[Service]
-Type=oneshot
-ExecStart=/usr/local/sbin/halogen-fabric-clock hold
-
-[Install]
-WantedBy=multi-user.target suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
@@ -23,7 +23,7 @@ After=gluetun.service network.target
 [Container]
 ContainerName=marreta_browser
 Network=gluetun.container
-Image=docker.io/lightpanda/browser:nightly@sha256:e72730e364d1fbf352c1149f29ffa4b278f3b1a55ff0dec3262298ebe624ec7a
+Image=docker.io/lightpanda/browser:nightly

 [Service]
 TimeoutStartSec=300
@@ -32,7 +32,7 @@ After=gluetun.service marreta-browser.service network.target
 [Container]
 ContainerName=marreta
 Network=gluetun.container
-Image=ghcr.io/manualdousuario/marreta:latest@sha256:30f2a34567b79b66da794b4b9423751bbbcd07ceff600d9f6a2d219e7e439362
+Image=ghcr.io/manualdousuario/marreta:latest
 # Relocate off SearXNG's 8080 inside the shared netns.
 Environment=NGINX_HTTP_PORT=8085
 Environment=APP_NAME=Marreta
@@ -33,7 +33,7 @@ Wants=network-online.target

 [Container]
 ContainerName=alertmanager
-Image=docker.io/prom/alertmanager:latest@sha256:39d8bb3966c694a124428a3a88fbcffd40bd069129ecd47c059f8a3f17c55f3b
+Image=docker.io/prom/alertmanager:latest
 Network=dankclaw_default
 User=0:0
 Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
@@ -1,51 +1,55 @@
-# Quadlet unit: alloy (ROOT system service, boot-persistent).
+# Quadlet unit: alloy (rootless, boot-persistent).
 #
-# Install path on dankclaw: /etc/containers/systemd/alloy.container
-# Unit name: alloy.service (container name: alloy) — SYSTEM manager, root.
+# Install path on dankclaw: ~/.config/containers/systemd/alloy.container
+# Unit name: alloy.service (container name: alloy)
 #
-# WHY ROOT (not rootless, unlike its observability siblings). Alloy reads the
-# host journal. The journal files are root:systemd-journal 0640 (other::---);
-# a rootless container carries no supplementary groups and cannot open them —
-# a rootless read of system.journal returns DENIED (see the Alloy page in the
-# vault). Running as root, one loki.source.journal reads system.journal AND
-# user-1000.journal, so it carries both host logs and the rootless Quadlet
-# containers' logs. Validated 2026-10-20: a root probe read 31,825 journal
-# lines and pushed 26,966 into Loki in 20 s, zero drops.
+# Log shipper. This replaces the compose service of the same name in
+# dankclaw/compose.yaml ("Observability" block).
 #
-# NETWORK=host. Loki publishes 127.0.0.1:3100 (loopback only) on the host, and
-# no bridge container — rootless or root — can reach the host's loopback, so
-# this unit runs on the host network and pushes to 127.0.0.1:3100. The side
-# effect: the UI must bind loopback too, so Exec sets the listen-addr to
-# 127.0.0.1 (the old rootless unit used 0.0.0.0 only because container loopback
-# was not host loopback; under Network=host they are the same address, so the
-# unauthenticated UI stays loopback-only — nothing off-host reaches it).
+# IMAGE. franksdank/alloy is built locally by deploy-remote.sh
+# (build_compose_image) from dankclaw/alloy/Containerfile: upstream
+# grafana/alloy plus a static busybox for the healthcheck. Pull=never keeps a
+# boot from reaching for that local tag at a registry.
 #
-# THE /etc/machine-id MOUNT IS LOAD-BEARING. sd_journal locates the journal
-# directory by the local machine id; without it the container opens nothing and
-# the journal source reads exactly 0 lines — silently, no error, health green.
-# This mount is the entire reason the pipeline carries data. Never remove it.
+# NETWORK. It joins dankclaw_default so its config's loki.write endpoint
+# resolves the container name loki:3100.
 #
-# IMAGE. Built rootless by build_compose_image alloy, then mirrored into the
-# root image store by the deployer (a root build runs in the build container's
-# own resolver and fails on DNS here). Pull=never: boot must not reach for the
-# local tag at a registry.
+# THE UI PORT EXISTS ON PURPOSE. Alloy's UI listens on the container's own
+# 127.0.0.1 by default and cannot be moved from the config file (alloy 1.x
+# refuses the ui block), so the compose service had no reachable health
+# endpoint at all: the homepage status card for alloy reported "down"
+# continuously and there was nothing for a deploy to wait on. The command
+# below therefore moves the UI to the container's wildcard address and the
+# port is published on host loopback only, which puts alloy in the same class
+# as the other three: reachable from the host, by Caddy over the private
+# network, and by nothing off-host. Alloy has no authentication, which is
+# exactly why it is not published wider.
+#
+# SOURCES. The host journald socket and the host log/container-storage trees
+# are bind-mounted read-only; Alloy tails them and pushes to Loki. The config
+# lives at /opt/alloy/config/config.alloy, installed from
+# shared/alloy/config.alloy on every deploy.
+#
+# ORDERING. Wants/After loki.service: without a reachable Loki, Alloy's write
+# client fails and its log pipeline backs up.

 [Unit]
 Description=Alloy log shipper (Dankclaw)
-After=network-online.target
+After=network-online.target loki.service
+Wants=network-online.target loki.service

 [Container]
 ContainerName=alloy
 Image=localhost/franksdank/alloy:latest
 Pull=never
-Network=host
-Exec=run --server.http.listen-addr=127.0.0.1:12345 /etc/alloy/config.alloy
-Volume=/opt/alloy/config:/etc/alloy:ro
-Volume=/etc/machine-id:/etc/machine-id:ro
+Network=dankclaw_default
+Exec=run --server.http.listen-addr=0.0.0.0:12345 /etc/alloy/config.alloy
+PublishPort=127.0.0.1:12345:12345
+Volume=/opt/alloy/config:/etc/alloy
 Volume=/run/systemd/journal/socket:/run/systemd/journal/socket:ro
-Volume=/var/log/journal:/var/log/journal:ro
-Volume=/run/log/journal:/run/log/journal:ro
-HealthCmd=/usr/local/bin/busybox wget --spider -q http://127.0.0.1:12345/-/healthy
+Volume=/var/log:/var/log:ro
+Volume=/var/lib/containers:/var/lib/containers:ro
+HealthCmd=/usr/local/bin/busybox wget --spider -q http://localhost:12345/-/healthy
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
@@ -56,4 +60,4 @@ Restart=always
 RestartSec=10

 [Install]
-WantedBy=multi-user.target
+WantedBy=default.target
@@ -1,56 +0,0 @@
-# Quadlet unit: blackbox exporter (rootless, host network, boot-persistent).
-#
-# Install path on dankclaw:
-#   ~/.config/containers/systemd/blackbox-exporter.container
-#
-# WHY Network=host. The probes it runs are (a) the host's own LAN IP
-# (192.168.0.143:8081 SearXNG, :3005 OpenWebUI — host-process/portmap binds a
-# bridge container cannot reach; netavark DNATs published ports only, see
-# Container Host Reachability in the vault) and (b) ComfyUI's published port
-# via 127.0.0.1 (rootless --network=host shares the real host netns, so the
-# 0.0.0.0:8188 publish answers on loopback). A bridge-networked prober could
-# reach neither class reliably.
-#
-# WHY the listener binds 127.0.0.1 + LAN. A bridge container CANNOT dial
-# its own host's LAN IP (netavark hairpin gap, measured 2026-10-21), but a
-# bridge container dialing the podman gateway 169.254.1.2 lands on the host
-# LOOPBACK — that is the scrape path, and it needs this exporter to be
-# LISTENING ON LOOPBACK (proved: 200 from the prometheus container to
-# http://169.254.1.2:9115/-/healthy with a loopback bind). The gateway
-# address itself is NOT bindable from the host netns — it exists only
-# inside each rootless netns; binding it dies "bind: cannot assign
-# requested address" (live 2026-10-21). The LAN bind stays for diagnostics.
-# No public DNS name, no guard chain: internal net only, per the exposure
-# policy. The listen flag is --web.listen-address and it is REPEATABLE, one
-# flag per bind; plain --listen-address does not exist and the exporter
-# exits in 100 ms with "unknown long flag" (live 2026-10-21).
-
-[Unit]
-Description=Blackbox exporter (HTTP probe prober)
-After=network-online.target
-
-[Container]
-Image=docker.io/prom/blackbox-exporter@sha256:ea1c7aa3ef3f307cd03e85f34abb604478ccb90b567fc690da0cd9776da76b13
-ContainerName=blackbox-exporter
-Network=host
-# The image's ENTRYPOINT is /bin/blackbox_exporter itself (proved live
-# 2026-10-21: `podman run IMAGE sh` dies "unexpected sh"), so Exec= carries
-# FLAGS ONLY — repeating the binary makes it a positional arg and kingpin
-# exits "unexpected /bin/blackbox_exporter".
-Exec=--config.file=/etc/blackbox_exporter/config.yml --web.listen-address=127.0.0.1:9115 --web.listen-address=192.168.0.143:9115
-Volume=/opt/blackbox/blackbox.yml:/etc/blackbox_exporter/config.yml:ro
-# Plain exec form like every sibling unit. Shell form (`/bin/sh -c "…"`)
-# is passed as one quoted token and quadlet then folds the following
-# --health-interval/--health-retries (and the image name) into the command
-# string — unit died twice that way, live 2026-10-21. wget only; this
-# busybox variant has no curl, and host-network wget answers on loopback.
-HealthCmd=wget -q -O /dev/null http://127.0.0.1:9115/-/healthy
-HealthInterval=30s
-HealthRetries=3
-
-[Service]
-Restart=always
-RestartSec=5
-
-[Install]
-WantedBy=default.target
@@ -1,56 +0,0 @@
-# Quadlet unit: cadvisor (rootless, boot-persistent) — DANKCLAW.
-#
-# Install path on dankclaw: ~/.config/containers/systemd/cadvisor.container
-# Unit name: cadvisor.service (container name: cadvisor)
-#
-# WHY THIS EXISTS. The app stack is ~34 rootless Podman containers and until
-# now nothing reported per-container CPU/memory — Grafana could only see the
-# host (node_exporter) and the LLM engines. cadvisor reads the host cgroup
-# tree and exports container_cpu_* / container_memory_* per container.
-#
-# THE LOAD-BEARING FLAG IS --cgroupns=host. Podman starts every container in a
-# PRIVATE cgroup namespace, so an exporter container sees only its OWN cgroup
-# subtree: cadvisor exported a single root series until this flag was added,
-# then it walked the whole tree (996 per-container series measured 2026-10-20).
-# The quadlet unit name is carried in the cgroup path — app.slice/<name>.service/
-# libpod-payload-<hex> — so Prometheus relabels that path into a `container`
-# label (shared/prometheus/prometheus.yml, cadvisor-dankclaw job) and no
-# libpod/podman socket is needed to name containers.
-#
-# BINDING. cadvisor has no per-interface listen flag (unlike node_exporter's
-# --web.listen-address), so it binds 0.0.0.0 INSIDE its own network namespace.
-# The bridge PublishPort is what keeps the unauthenticated endpoint off the
-# host's other interfaces: it publishes exactly loopback (host-local probes and
-# the deployer's health gate) and this host's LAN address (Prometheus, a bridge
-# container, reaches a host-published port at its LAN address — the same netavark
-# rule node-exporter relies on, only here the port really is published).
-#
-# It is rootless on purpose: dank owns the app containers' cgroups, so a
-# rootless cadvisor reads every one of them; root would buy nothing here.
-
-[Unit]
-Description=cAdvisor container metrics (Dankclaw)
-After=network-online.target
-Wants=network-online.target
-
-[Container]
-ContainerName=cadvisor
-Image=gcr.io/cadvisor/cadvisor:v0.49.1@sha256:3cde6faf0791ebf7b41d6f8ae7145466fed712ea6f252c935294d2608b1af388
-PodmanArgs=--pid=host --cgroupns=host
-PublishPort=127.0.0.1:8080:8080
-PublishPort=192.168.0.143:8080:8080
-Volume=/sys:/sys:ro
-Volume=/:/rootfs:ro
-Exec=--housekeeping_interval=30s
-HealthCmd=wget --spider -q http://localhost:8080/healthz
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=40s
-
-[Service]
-Restart=always
-RestartSec=10
-
-[Install]
-WantedBy=default.target
@@ -34,7 +34,7 @@ Wants=network-online.target prometheus.service

 [Container]
 ContainerName=grafana
-Image=docker.io/grafana/grafana:latest@sha256:488984b716c0862639f91a698adbe2862677a5a07bad42da5f03f3030883b205
+Image=docker.io/grafana/grafana:latest
 Network=dankclaw_default
 User=0:0
 PublishPort=127.0.0.1:3002:3000
@@ -47,24 +47,9 @@ Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=${GRAFANA_OAUTH_CLIENT_SECRET}
 # client_id query parameter — /application/o/<slug>/authorize/ does not exist.
 Environment=GF_AUTH_GENERIC_OAUTH_AUTH_URL=https://auth.franksdank.cloud/application/o/authorize/
 Environment=GF_AUTH_GENERIC_OAUTH_TOKEN_URL=https://auth.franksdank.cloud/application/o/token/
-# Standard OIDC userinfo. Grafana v13 fetches <api_url>/emails for the address
-# ONLY when the token carries no `email` claim (authentik has no /emails path and
-# 404s it, failing the login) — so the claim must actually be there: the fix is
-# the grafana-email-scope mapping (see scripts/authentik-grafana-sso-fix.sh),
-# not a Grafana flag. Do NOT append ?api=1 to this URL: Grafana then GETs
-# <...>/userinfo/?api=1/emails, a 403.
-Environment=GF_AUTH_GENERIC_OAUTH_API_URL=https://auth.franksdank.cloud/application/o/userinfo/
-# authentik's scopes are openid/email/profile. `user:email` (a GitLab-ism) is
-# not a known scope here; requesting it makes the authorize view crash.
-# Quoted: quadlet word-splits Environment=, so an unquoted spaced list
-# truncates SCOPES to its first token.
-Environment="GF_AUTH_GENERIC_OAUTH_SCOPES=openid email profile"
-Environment=GF_AUTH_GENERIC_OAUTH_EMAIL_ATTRIBUTE_PATH=email
-Environment=GF_AUTH_GENERIC_OAUTH_LOGIN_ATTRIBUTE_PATH=sub
-Environment=GF_AUTH_GENERIC_OAUTH_NAME_ATTRIBUTE_PATH=preferred_username
-# First SSO login auto-provisions the user. Safe: unauthenticated traffic never
-# reaches this callback — the Anubis -> Authentik guard fronts every request.
-Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=true
+# ?api=1 = introspection-style userinfo: returns custom claims (groups) too.
+Environment=GF_AUTH_GENERIC_OAUTH_API_URL=https://auth.franksdank.cloud/application/o/userinfo/?api=1
+Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=false
 Environment=GF_AUTH_GENERIC_OAUTH_AUTO_UPSERT=true
 Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_ASSIGN_ORG=true
 Environment=GF_AUTH_GENERIC_OAUTH_ORG_ID=1
@@ -33,7 +33,7 @@ Wants=network-online.target

 [Container]
 ContainerName=node-exporter
-Image=docker.io/prom/node-exporter:latest@sha256:1b4e4438faca4dd7e001dd445d161a4a2091b0fededa84093b3a8dfeae1f1be0
+Image=docker.io/prom/node-exporter:latest
 Network=host
 PodmanArgs=--pid=host
 Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.143:9100
@@ -43,16 +43,8 @@ Wants=network-online.target

 [Container]
 ContainerName=prometheus
-Image=docker.io/prom/prometheus:latest@sha256:fa3302279710be2f21a8301ec17a8b4f2301fd829bd113839b3b6b02b6f191a2
+Image=docker.io/prom/prometheus:latest
 Network=dankclaw_default
-# Same-host scrape targets are dialed at host.containers.internal (the podman
-# gateway): a bridge container cannot dial its own host's LAN IP (netavark
-# hairpin gap, measured 2026-10-21), and traffic to the gateway lands on host
-# loopback / published ports (cadvisor/gpu jobs and the blackbox scrape all
-# ride it). The name resolves natively — NO HostAddr key: quadlet 5.7.0 calls
-# it unsupported and the WHOLE user quadlet batch then fails to generate,
-# silently orphaning every other unit (live 2026-10-21: prometheus.service
-# vanished while its container kept running as an orphan).
 User=0:0
 Exec=--config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --storage.tsdb.retention.time=30d --web.enable-lifecycle
 PublishPort=127.0.0.1:9090:9090
@@ -1,93 +0,0 @@
-# Quadlet unit: open-notebook's SurrealDB (rootless, boot-persistent).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/open-notebook-surrealdb.container
-# Unit name: open-notebook-surrealdb.service (container name: open-notebook-surrealdb)
-#
-# WHAT IT IS. open-notebook's only datastore: notebooks, notes, sources,
-# provider credentials and podcast episodes all live here. The app talks to it
-# over the WebSocket RPC on the private bridge (SURREAL_URL in
-# scripts/open-notebook.env), which is why the container NAME is part of the
-# contract — renaming it breaks the app's database URL, not just this file.
-#
-# IMAGE. `surrealdb/surrealdb:v2` pinned by manifest-list digest, resolved from
-# the registry on 2026-10-18 (that digest is SurrealDB 2.7.0, image built
-# 2026-09-23; `v2` itself is a MOVING tag, so the digest — not the tag — is what
-# makes this reproducible). Upstream open-notebook's own compose file pins the
-# same `v2` tag, and its Dockerfile copies /surreal out of the same image for
-# the retired single-container variant, so v2 is the version this app is built
-# against. Pull=never + the documented pull step (docs: [[Open Notebook]]):
-#
-#   podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
-#
-# NO PUBLISHED PORT. 8000 is deliberately not published: the app reaches it by
-# name on systemd-open-notebook, and a published 8000 would put the RPC with
-# the root credentials on every interface (upstream publishes 127.0.0.1:8000
-# "purely for local debugging" — a `podman exec` probe is enough here). The
-# binary binds 0.0.0.0:8000 INSIDE its own namespace via the image's
-# SURREAL_BIND env; without that image default it would bind loopback only and
-# the app could never connect, so no --bind flag belongs here.
-#
-# CREDENTIALS, AND WHY THEY ARE NOT IN THE COMMAND LINE. The `surreal start`
-# CLI reads its options from the environment (`--username` is [env:
-# SURREAL_USER=], `--password` is [env: SURREAL_PASS=] — verified against the
-# pinned image's own `start --help`), so the credentials arrive through the two
-# EnvironmentFile lines and the argv stays clean. Passing `--pass` instead —
-# what upstream's compose does — would put the database password in
-# /proc/<pid>/cmdline, readable by any `ps` on the host. Note the two
-# spellings: the DATABASE reads SURREAL_PASS, the APP reads SURREAL_PASSWORD,
-# and scripts/open-notebook.env declares both because they must hold the same
-# value; the deployer writes that pair when it seeds the host secret file.
-#
-# NO USER= OVERRIDE. The image runs as uid 65532 (Chainguard glibc-dynamic
-# base). With a NAMED volume that works unmodified: podman creates the volume's
-# backing directory owned by the container's user (measured with this digest —
-# the host directory lands on the mapped subuid and the store starts clean),
-# which is exactly why upstream's `user: root` line is not repeated here: that
-# line is commented in upstream as being "required for bind mounts", and this
-# unit has none. The consequence is that the DB files are NOT readable by dank
-# without `podman unshare` — accepted for a datastore nobody edits by hand;
-# episode AUDIO is a host directory for the opposite reason (see the app unit).
-#
-# HEALTH. The base image is distroless — measured against this digest, not even
-# `ls` exists (`exec: "ls": executable file not found in $PATH`), so there is no
-# `/bin/sh`, no curl and no wget. SurrealDB ships its own probe instead:
-# `surreal is-ready` connects to ws://localhost:8000 by default, needs no
-# credentials, prints `OK` and exits 0 while the server is up.
-#
-# It must be handed over as a QUADLET-SIDE JSON ARRAY, not a bare command.
-# podman wraps a bare `HealthCmd` string as `CMD-SHELL` (measured on this host's
-# podman 5.7: `--health-cmd "/surreal is-ready"` stores
-# `{"Test":["CMD-SHELL","/surreal is-ready"]}`), and with no `/bin/sh` in the
-# image that probe fails in ~30 ms with empty output on every interval while the
-# database answers perfectly — which is exactly how this service's first deploy
-# died on 2026-10-19 (`did not become healthy in 90s`). The quoted array survives
-# the generator (`--health-cmd "[\"/surreal\",\"is-ready\"]"`) and stores
-# `["CMD","/surreal","is-ready"]`, which execs directly and goes healthy. Do not
-# "simplify" it back to a bare command.
-
-[Unit]
-Description=SurrealDB (open-notebook datastore)
-After=network-online.target
-Wants=network-online.target
-
-[Container]
-ContainerName=open-notebook-surrealdb
-Image=docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
-Pull=never
-Network=open-notebook.network
-EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
-EnvironmentFile=/home/dank/Containers/open-notebook/env
-Exec=start --log info rocksdb:/mydata/mydatabase.db
-Volume=open-notebook-db:/mydata
-HealthCmd=["/surreal","is-ready"]
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=20s
-
-[Service]
-Restart=always
-RestartSec=5
-
-[Install]
-WantedBy=default.target
@@ -1,105 +0,0 @@
-# Quadlet unit: open-notebook (rootless, boot-persistent).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.container
-# Unit name: open-notebook.service (container name: open-notebook)
-#
-# WHAT IT IS. The fleet's podcast generator: notes and sources go in, a
-# two-voice (up to four) episode MP3 comes out. One image runs THREE processes
-# under supervisord — the FastAPI API on 5055, the background worker that
-# writes and speaks the episode, and the Next.js UI on 8502 — so this one unit
-# owns the whole app, and the frontend refuses to start until the API answers
-# (the image's own wait-for-api.sh). Runbook: [[Open Notebook]].
-#
-# IMAGE. `lfnovo/open_notebook:v1-latest` pinned by manifest-list digest,
-# resolved from the registry on 2026-10-18; it is the tag upstream's own
-# docker-compose.yml pins for the multi-container shape. Do NOT reach for the
-# `-single` tags: that variant bundles SurrealDB inside the app image, is
-# deprecated upstream, and would give this host two databases. Pull=never, so
-# the pin is fetched by the documented one-time step (docs: [[Open Notebook]]),
-# which the deployer also runs when the image is absent:
-#
-#   podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
-#
-# NETWORKS, in this order on purpose. The generated run line keeps file order,
-# so the DEFAULT ROUTE rides dankclaw_default: that is the network where this
-# host has measured that a bridge container can reach a host-namespace
-# listener through the gateway name (the Prometheus scrape of frank-zealot is
-# the proof — see [[Container Network Namespaces]]), and frank-tts lives there
-# too, so the TTS credential can be `http://frank-tts:7772/v1`. The private
-# systemd-open-notebook bridge is second and carries only the database; its
-# traffic is on-link, so losing the default route costs it nothing.
-#
-# PUBLISHED PORTS, and the 5055 collision. Upstream publishes 8502 and 5055,
-# and 5055 is exactly the port seerr already holds on this host
-# (`PublishPort=127.0.0.1:5055:5055`; `ss -ltn` on Dankclaw 2026-10-18 shows
-# 127.0.0.1:5055 taken). So the API leaves the container on host port 5056.
-# Only 8502 is published on all interfaces, because it is the single upstream
-# the guard chain dials: notebook.franksdank.cloud → anubis → authentik →
-# http://:8017 → 192.168.0.143:8502 (dankclaw/Caddyfile), and this Caddy lives
-# in root's storage, so a published port is the only way it can reach a rootless
-# quadlet. The API needs no outward face at all — the image's own Next.js layer
-# rewrites /api/* to the FastAPI beside it, so one origin serves UI and API —
-# and the fleet's rule is that loopback-only is for services whose user path is
-# the guard chain, which this became on 2026-10-19; 5056 stays published on
-# loopback because the deployer's health gate dials it there. That is also why
-# API_URL in scripts/open-notebook.env names the vhost rather than a port: left
-# to itself the frontend auto-detects `<host-you-opened>:5055`, and the UI loads
-# while every call fails. OPEN_NOTEBOOK_PASSWORD is not made redundant by the
-# chain: it is what keeps the API honest for everything else on this host, and
-# with it unset the API answers anyone, including the unauthenticated
-# episode-audio download.
-#
-# VOLUMES. `/app/data` is a named volume: uploads, the LangGraph SQLite
-# checkpoints and the extraction caches the image keeps under it
-# (UV_CACHE_DIR, HF_HOME) are app state nobody browses. The ONE exception is
-# finished audio, which is bind-mounted out of the container at
-# /home/dank/media/podcasts. That mount is interim and reversible: episode
-# destination is an open human-only decision (Gitea issue #7, label
-# `human-only` — agents must not work it), and until it lands, episodes are
-# ordinary files on the host. DATA_FOLDER is hard-coded `./data` in the app
-# (WORKDIR /app) and the DB stores audio paths RELATIVE to data/podcasts, so
-# covering that one subdirectory is enough and removing the line changes
-# nothing else. The image declares no USER, so the app runs as container root,
-# which in this host's rootless mapping (0 -> 1000, measured on this host —
-# see [[Seerr]]) writes those episode files as dank.
-#
-# HEALTH. `GET /health` on the API returns {"status":"healthy"} and is in the
-# password middleware's excluded-path list, so the probe stays honest after the
-# password is set (both read from api/main.py upstream). The image installs
-# curl and ffmpeg in the runtime layer, so the probe needs no shell — kept
-# argument-only for the generator's space-splitting. Note what this probe does
-# NOT prove: the route returns a static body, so it says the API serves, not
-# that the database answers. The DB has its own probe in its unit, and the
-# worker's failure to reach the database shows up as failed jobs, not as an
-# unhealthy container.
-
-[Unit]
-Description=open-notebook (podcast generator: API, worker, web UI)
-After=network-online.target open-notebook-surrealdb.service
-Wants=network-online.target open-notebook-surrealdb.service
-
-[Container]
-ContainerName=open-notebook
-Image=docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
-Pull=never
-Network=dankclaw_default
-Network=open-notebook.network
-PublishPort=0.0.0.0:8502:8502
-PublishPort=127.0.0.1:5056:5055
-EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
-EnvironmentFile=/home/dank/Containers/open-notebook/env
-Volume=open-notebook-data:/app/data
-Volume=/home/dank/media/podcasts:/app/data/podcasts
-HealthCmd=curl -fsS http://127.0.0.1:5055/health
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=60s
-
-[Service]
-Restart=always
-RestartSec=10
-TimeoutStartSec=900
-
-[Install]
-WantedBy=default.target
@@ -1,34 +0,0 @@
-# Quadlet network: private bridge for the open-notebook pair (rootless).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.network
-# Unit name: open-notebook-network.service (podman network name:
-# systemd-open-notebook — quadlet prefixes "systemd-", so probe the runtime
-# name, never the filename; see [[Container Network Namespaces]]).
-#
-# WHY A PRIVATE BRIDGE. The app reaches its database by container name
-# (SURREAL_URL=ws://open-notebook-surrealdb:8000/rpc), and container DNS names
-# resolve only on the network that owns them. Putting SurrealDB here instead of
-# on dankclaw_default keeps the datastore reachable ONLY to the container that
-# owns it: 8000 is not published (an unpublished port is refused outright from
-# every other namespace anyway), and nothing else on the fleet-wide network can
-# dial it with the root credentials. Same reasoning as the gitea and authentik
-# database pairs.
-#
-# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
-# per-container userspace helper left a container that accepted TCP and reset
-# every request for two days, logging nothing). A bridge has no such helper.
-#
-# Subnet registry on dankclaw (rootless):
-#   10.88.0.0/16  default "podman" bridge
-#   10.89.0.0/24  gitea (systemd-gitea)
-#   10.89.1.0/24  compose stack (dankclaw_default)
-#   10.89.2.0/24  searxng (systemd-searxng)
-#   10.89.3.0/24  open-webui (systemd-open-webui)
-#   10.89.4.0/24  open-notebook (systemd-open-notebook)  <- this file
-# Pinned so the container IPs stay deterministic across reboots, like the
-# other pinned nets; 10.89.4.0/24 was the next unclaimed /24 in this registry.
-
-[Network]
-Driver=bridge
-Subnet=10.89.4.0/24
-Gateway=10.89.4.1
@@ -27,7 +27,7 @@ After=network.target
 [Container]
 ContainerName=open-webui
 Network=open-webui.network
-Image=ghcr.io/open-webui/open-webui:main@sha256:0b399a8ba34558e181d316750455f2a95376a38cd60a61b9c942e138898944bd
+Image=ghcr.io/open-webui/open-webui:main
 PublishPort=3005:8080
 Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
 Environment=ANONYMIZED_TELEMETRY=False
@@ -45,4 +45,3 @@
 192.168.0.143 pihole.franksdank.cloud
 192.168.0.143 seerr.franksdank.cloud
 192.168.0.143 marreta.franksdank.cloud
-192.168.0.143 notebook.franksdank.cloud
@@ -24,7 +24,7 @@ Wants=network-online.target

 [Container]
 ContainerName=pihole
-Image=docker.io/pihole/pihole:latest@sha256:d34134716fc32038b052a8674c7c6879e0ff21f0e6d8af345166fffa2c27c113
+Image=docker.io/pihole/pihole:latest
 Network=slirp4netns
 PublishPort=0.0.0.0:53:53/udp
 PublishPort=0.0.0.0:53:53/tcp
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
 [Container]
 ContainerName=gluetun
 Network=searxng.network
-Image=docker.io/qmcgaw/gluetun:latest@sha256:4af0c804003481e100180e1f8d4d5e9c48163fedc0f88d061881b615aef9d183
+Image=docker.io/qmcgaw/gluetun:latest
 AddCapability=NET_ADMIN
 AddDevice=/dev/net/tun:/dev/net/tun
 Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
 ContainerName=searxng-valkey
 Network=searxng.network
 NetworkAlias=valkey
-Image=docker.io/valkey/valkey:9-alpine@sha256:bf09d58e96d6b4524a15396b65b242bbe18beebbcc15a04d8ee75ddac62fae0b
+Image=docker.io/valkey/valkey:9-alpine
 Volume=/opt/searxng/valkey:/data

 [Service]
@@ -16,7 +16,7 @@ After=gluetun.service searxng-valkey.service network.target
 [Container]
 ContainerName=searxng
 Network=gluetun.container
-Image=ghcr.io/searxng/searxng:latest@sha256:f4aa660c54d062dedb268be84e821ea24e39f48c7a496af8dd02cc9eedb93847
+Image=ghcr.io/searxng/searxng:latest
 Environment=SEARXNG_SECRET=${SEARXNG_SECRET}
 Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL}
 Volume=/opt/searxng/config:/etc/searxng
@@ -29,7 +29,7 @@ Wants=network-online.target

 [Container]
 ContainerName=seerr
-Image=ghcr.io/seerr-team/seerr:v3.5.1@sha256:d8fab675b55ad293ded98fd71efcfd46054cbad9746534ae1cc4794a88c0b3be
+Image=ghcr.io/seerr-team/seerr:v3.4.1
 Network=dankclaw_default
 PublishPort=127.0.0.1:5055:5055
 Volume=/home/dank/Containers/seerr/data:/app/config
@@ -1,26 +0,0 @@
-# /etc/udev/rules.d/98-halogen-npu-access.rules
-#
-# Why the NPU node is world-writable: `frank-dragoon` runs rootless podman under a
-# unit with `User=dank`, and the container's user namespace maps exactly one uid and
-# one gid (dank). `/dev/accel/accel0` is `root:render 0660`, so inside the container
-# the node reads `nobody:nogroup` and the engine dies with its own diagnosis:
-#
-#   halogen npu: /dev/accel/accel0 is here, but this process cannot open it
-#     (crw-rw---- nobody:nogroup).
-#
-# `--group-add keep-groups` is the vendor's advice and it does NOT work in this
-# shape: measured 2026-10-19 from the host as a `dank` shell that was already in
-# `render`, with and without the flag, the node still read `nobody:nogroup` and the
-# open was refused — an unmapped gid cannot be granted to a process inside a
-# namespace that does not map it. Same conclusion `99-comfyui-gpu-access.rules`
-# reached for `/dev/kfd` from the systemd-user-spawn direction: the mode is the only
-# spawn-context-independent door short of running the stack rootful.
-#
-# Blast radius: any process in this host's single uid domain (every rootless
-# container here shares it) can submit to the XDNA2 NPU. The NPU holds no user data
-# and no privileged context — the same trade already accepted for the GPU compute
-# nodes beside it. Installed by `deploy-remote.sh` when the NPU lane is armed and left
-# in place afterwards — the mode costs nothing while the lane is idle. `serve-frank-dragoon.sh`
-# refuses to arm the lane on a node that is not other-accessible, so a host that lost this
-# file fails loudly at boot instead of crash-looping.
-KERNEL=="accel[0-9]*", SUBSYSTEM=="accel", MODE="0666"
@@ -1,43 +0,0 @@
-# frank-tts — the cloned-voice speech façade (dankclaw/voiceservice/app.py).
-#
-# Standard library only, so there is nothing to pip-install and nothing to
-# compile: the image is a python runtime plus ffmpeg (the one binary the app
-# shells out to, for response_format=mp3) plus one COPY of app.py. Rebuild to
-# ship a new app.py.
-#
-# Build on dankclaw (deploy-remote.sh build_compose_image frank-tts does this,
-# including --format docker, which the fleet's other local builds also pass):
-#   podman build --format docker -t franksdank/frank-tts:latest \
-#       -f dankclaw/voiceservice/Containerfile .
-#
-# BASE PINNED BY DIGEST, like every other image this fleet runs from a
-# registry: a tag moves under a rebuild and a voice endpoint that changes
-# python or ffmpeg mid-week is a debug session nobody asked for. The digest is
-# the multi-arch INDEX of python:3.12-alpine as published 2026-10-18 (amd64 +
-# arm64 on linux); Dankclaw resolves the amd64 entry.
-#
-# WHY 3.12: the fleet's ceiling for this app is python <= 3.12 (single file,
-# no build step); the alpine flavour also keeps the image in the tens of MB
-# and ships busybox `wget`, which is what the quadlet unit's HealthCmd runs —
-# so no HEALTHCHECK belongs here (the probe lives in the unit, as in
-# alertmanager/prometheus; only distroless bases need it baked in).
-FROM docker.io/library/python:3.12-alpine@sha256:1b668429b3511ab407d8e00648891631b0b1a4d7e15e3ca70f38ab5b91ad4ab4
-
-# ffmpeg: mp3 output. The app always asks the upstream for wav and re-encodes
-# on the way out, so ffmpeg is the only external program in the picture.
-RUN apk add --no-cache ffmpeg
-
-# Unbuffered stdout: the unit's journal must show the "listening on" line (and
-# any backend error) the moment it happens, not when the buffer fills.
-ENV PYTHONUNBUFFERED=1
-
-COPY dankclaw/voiceservice/app.py /app/app.py
-
-# Non-root. The voice packs are bind-mounted read-only, so this user only ever
-# reads them — they must be world-readable (the pack builder writes 644/755).
-# The one thing it writes is its per-request temp dir under /tmp.
-RUN adduser -D -u 10000 -h /app -s /sbin/nologin shim
-USER shim
-
-EXPOSE 7772
-CMD ["python3", "/app/app.py"]
@@ -1,178 +0,0 @@
-# frank-tts-engine — the arm A TTS engine: qwentts.cpp's resident `tts-server`
-# (the CPU binary that frank-tts on :7772 forwards to). The runbook is
-# [[Frank TTS Engine]]; the fleet-facing endpoint is [[Frank TTS]].
-#
-# WHAT THIS IS. The upstream source is NOT vendored here: the image clones
-# ServeurpersoCom/qwentts.cpp at one pinned commit and builds it. Rebuilding is
-# therefore a network operation (github.com plus objects.githubusercontent.com
-# for the `ggml` submodule), and build_compose_image in scripts/deploy-remote.sh
-# skips it whenever this Containerfile's sha256 label already matches the built
-# image — the same offline-safe idiom as loki/alloy/comfyui, so a re-deploy of
-# an unchanged engine needs no network.
-#
-# THE PINNED COMMIT IS THE CONTRACT. 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
-# is the commit the 2026-10-18 CPU trial on Dankclaw measured (server-side TTFA
-# 94-112 ms, warm RTF 0.55-0.67, run spread +-0.01 — the numbers that won the
-# engine decision). Moving it means re-measuring, not just editing this line.
-# `ggml` is a submodule of the repo (the only one; vendor/cpp-httplib and
-# vendor/yyjson are ordinary directories in the tree), and the submodule commit
-# is recorded in the parent tree, so cloning the parent commit recursively is
-# reproducible without a second pin.
-#
-# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim — with the Vulkan door
-# open since 2026-10-19. That configure was run in this exact base image and
-# answers `Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version
-# "1.3.275") found components: glslc` then `Including Vulkan backend`:
-#
-#   cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
-#       -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
-#       -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON \
-#       -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
-#   && cmake --build build -j 8
-#
-# Every flag is load-bearing, so do not "tidy" it:
-#   GGML_BLAS=OFF is FORCED, not a preference — Dankclaw has libopenblas.so.0
-#     but no BLAS dev headers, so the upstream default (BLAS on) fails to
-#     configure. Upstream's own Dockerfile and buildcpu.sh both pass BLAS on,
-#     which is why this file does not copy them.
-#   GGML_CCACHE=OFF because a container build layer has no persistent ccache
-#     directory to hit; on, it only writes a useless one.
-#   GGML_OPENMP=ON is what makes the runtime need libgomp1 (installed below).
-#   GGML_BACKEND_DL=OFF / GGML_CPU_ALL_VARIANTS=OFF: upstream's images build
-#     every x86 ISA as a loadable backend so one image runs anywhere. This one
-#     is built on the machine that runs it, so the single native CPU backend is
-#     what was measured. That is also why no -DGGML_NATIVE is passed: ggml
-#     defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
-#     host — correct while build host == run host, wrong if anyone ever builds
-#     this image elsewhere, which is why the fleet builds it on Dankclaw.
-#   CUDA/HIP/METAL/SYCL/RPC stay OFF, VULKAN is ON: the endpoint computes on the
-#     gfx1151 iGPU (decision and co-tenancy gates:
-#     report/tts-npu-spike-verdict-2026-10-19.md, shape in [[Frank TTS Engine]]).
-#     Nothing else is available to this image: no ROCm userland, no NVIDIA
-#     userland, no macOS, no oneAPI, and RPC would add a second process to
-#     supervise on a box that already has one engine per card.
-#
-# BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
-# tts-server links libggml*.so rather than swallowing them; the runtime stage
-# copies the binary and its shared objects out of the build tree and points
-# LD_LIBRARY_PATH at them, exactly as upstream's own Dockerfile does (the RPATH
-# baked at link time names the build directory, which does not exist here).
-#
-# BASE PINNED BY DIGEST, like every other image this fleet builds: the digest
-# is the multi-arch INDEX of ubuntu:24.04 as resolved from Docker Hub on
-# 2026-10-18; Dankclaw takes the amd64 entry. One base for both stages so there
-# is exactly one thing to re-resolve.
-#
-# NO HEALTHCHECK IN THIS FILE. The probe lives in the Quadlet unit
-# (HealthCmd=), which is the fleet pattern for anything with a shell — see the
-# alertmanager/prometheus units, and the opposite case in dankclaw/loki, whose
-# distroless image has no /bin/sh for the generator's CMD-SHELL wrapping.
-# `curl` is therefore installed in the runtime stage, for that probe only.
-
-ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac
-
-# ------------------------------------------------------------------ build
-FROM ${BASE_IMAGE} AS build
-
-# The three Vulkan build deps are not optional and not interchangeable:
-#   libvulkan-dev    vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake
-#                    configs, i.e. what find_package(Vulkan) resolves to.
-#   glslc            ggml compiles its shaders with glslc, not glslangValidator.
-#                    Ubuntu's glslang-tools ships glslangValidator and NOT glslc,
-#                    so without this package `find_package(Vulkan COMPONENTS
-#                    glslc REQUIRED)` fails and there is no Vulkan backend.
-#                    The archive's glslc is old (2023.8): it compiles
-#                    GL_KHR_cooperative_matrix — the path ggml's quantized
-#                    matmul needs — but reports cooperative_matrix2,
-#                    integer_dot_product, bfloat16 and the micro-float
-#                    extensions as unsupported, so those shader families are
-#                    compiled out. If a Vulkan prefill ever reads as
-#                    inexplicably slow, suspect this compiler before the device;
-#                    the fix is a pinned glslang build passed as
-#                    -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
-#   spirv-headers    find_package(SPIRV-Headers CONFIG REQUIRED); the distro
-#                    package ships /usr/share/cmake/SPIRV-Headers/.
-RUN apt-get update -qq \
-    && apt-get install -y -qq --no-install-recommends \
-        ca-certificates cmake g++ git make \
-        libvulkan-dev glslc spirv-headers \
-    && rm -rf /var/lib/apt/lists/*
-
-# Checked out detached at a full SHA, then the recorded submodule. No branch
-# name anywhere: a branch moves, this commit does not.
-ARG QWENTTS_COMMIT=5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
-RUN git clone --no-checkout https://github.com/ServeurpersoCom/qwentts.cpp /src \
-    && git -C /src checkout --detach "${QWENTTS_COMMIT}" \
-    && git -C /src submodule update --init --depth 1 \
-    && git -C /src rev-parse HEAD \
-    && git -C /src submodule status
-
-WORKDIR /src
-RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
-        -DGGML_CCACHE=OFF \
-        -DGGML_BACKEND_DL=OFF \
-        -DGGML_CPU_ALL_VARIANTS=OFF \
-        -DGGML_OPENMP=ON \
-        -DGGML_BLAS=OFF \
-        -DGGML_CUDA=OFF \
-        -DGGML_HIP=OFF \
-        -DGGML_VULKAN=ON \
-        -DGGML_METAL=OFF \
-        -DGGML_SYCL=OFF \
-        -DGGML_RPC=OFF \
-    && cmake --build build -j 8
-
-# ---------------------------------------------------------------- runtime
-FROM ${BASE_IMAGE}
-
-# libgomp1: the OpenMP runtime the build above links against. curl: the unit's
-# HealthCmd. ca-certificates: so that curl can speak https if anyone ever
-# points a probe at a URL with a scheme this host terminates elsewhere.
-#
-# libvulkan1 + mesa-vulkan-drivers are what make `--device Vulkan0` mean
-# anything at runtime: libvulkan1 is the loader libggml-vulkan.so dlopens, and
-# mesa-vulkan-drivers is the RADV ICD that claims the gfx1151. Loader without an
-# ICD enumerates ZERO devices, and tts-server then dies in device lookup AFTER
-# both GGUFs have been read — the journal reads as a model fault, not a missing
-# driver. Enumerated from this base on 2026-10-19 with `--device /dev/dri
-# --group-add keep-groups` plus the host's ComfyUI udev rule: "Radeon 8060S
-# Graphics (RADV GFX1151)", API 1.4.318 (the heap figure is owned by
-# [[Frank TTS Engine]]; it is the GTT-backed aperture, not the ~2 GiB DRM
-# carve-out). No NVIDIA ICD is installed on purpose: one Vulkan device in the
-# enumeration is what makes the NAME Vulkan0 safe to write down.
-RUN apt-get update -qq \
-    && apt-get install -y -qq --no-install-recommends \
-        ca-certificates curl libgomp1 libvulkan1 mesa-vulkan-drivers \
-    && rm -rf /var/lib/apt/lists/*
-
-# Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
-# model directory are host state mounted read-only, and one uid across the pair
-# means one thing to remember about what is world-readable.
-#
-# `useradd`, NOT `adduser`: this trimmed ubuntu:24.04 base ships no `adduser` at
-# all (verified 2026-10-18 against this digest — `adduser` is absent from the
-# base and is not pulled in by ca-certificates/curl/libgomp1 either), so an
-# `adduser` line dies at image-build time with crun's "executable file `adduser`
-# not found in $PATH". `useradd --system --uid 10000` succeeds and prints one
-# cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`); the uid is the
-# point, and it is what matches the façade. The alpine façade Containerfile keeps
-# its own busybox `adduser -D -u 10000` — different base, different tool, same uid.
-RUN useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts
-
-WORKDIR /app
-COPY --from=build /src/build/tts-server /src/build/*.so* /app/
-COPY dankclaw/voiceservice/tts-engine-start.sh /app/start.sh
-# The boot warmer lives INSIDE the image because the engine publishes no port:
-# the unit reaches it with `podman exec`, and the script needs the same curl,
-# base64 and /voices the engine's own namespace has. It is a sibling of
-# start.sh, not a second entrypoint — CMD stays the server.
-COPY dankclaw/voiceservice/tts-engine-warm.sh /app/warm-voices.sh
-RUN chmod 755 /app/start.sh /app/warm-voices.sh
-
-# The binary's RPATH points inside the build tree; this is what makes it find
-# the libggml*.so copied beside it.
-ENV LD_LIBRARY_PATH=/app
-
-USER tts
-EXPOSE 8880
-CMD ["/app/start.sh"]
@@ -1,424 +0,0 @@
-#!/usr/bin/env python3
-"""frank-tts — the fleet's OpenAI-compatible speech façade (component: the shim).
-
-One local endpoint that answers ``voice=<name>`` with audio in a cloned voice.
-Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
-``POST /v1/audio/speech`` shape and never learn a file path: a voice is a
-directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
-the name real. A name that has no reference is not a voice — the shim answers
-400 with the list that is, instead of silently narrating in a fallback voice
-(silent fallback is how a wrong-voice endpoint stays wrong for weeks). A request
-that names NO voice at all takes ``$DEFAULT_VOICE``: that is a spoken choice in
-the env file rather than a surprise in the code, and the journal marks it
-``(default)`` so a name nobody asked for is never mistaken for one that was.
-
-Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
-
-``omni-http``
-    Forward to an OpenAI-shaped upstream TTS server (``$TTS_UPSTREAM``) and
-    carry the cloned voice with the request as ``reference_audio`` (base64 of
-    ``reference.wav``). The upstream does the inference; this process only
-    resolves the voice and, when asked, re-encodes to mp3 with ``ffmpeg``.
-
-``llama-tts-cli``
-    Run a local binary per request (``$LLAMA_TTS_BIN``), hand it the
-    reference with ``--tts-speaker-file``, read the WAV it writes. ``nice -n
-    19`` because this box also serves text and a voice request must never
-    starve it.
-
-Everything comes from the environment, read at REQUEST time (no
-import-time-only globals): the same module can be imported with an injected
-``Config`` or run with ``python3 app.py`` and the unit's env file.
-
-Runtime law: single process, standard library only, no build step. Audio
-bytes never reach the log — request lines and error text only.
-"""
-
-from __future__ import annotations
-
-import argparse
-import base64
-import json
-import os
-import re
-import subprocess
-import sys
-import tempfile
-import urllib.error
-import urllib.parse
-import urllib.request
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-
-WAV_CONTENT_TYPE = "audio/wav"
-MP3_CONTENT_TYPE = "audio/mpeg"
-
-# A voice name is a plain directory name. Slashes and `..` are refused here so
-# a client-supplied name can never escape $VOICES_DIR.
-VOICE_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
-
-
-class ShimError(Exception):
-    """A failure with the HTTP status the client should see."""
-
-    def __init__(self, status, message):
-        super().__init__(message)
-        self.status = status
-        self.message = message
-
-
-class Config:
-    """Every knob of the shim. Constructed from the environment per request by
-    default, or injected into ``create_server`` for tests and embedding."""
-
-    def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
-                 llama_model, llama_mmproj, llama_lang, llama_threads, port,
-                 max_chars, timeout, default_voice=""):
-        self.voices_dir = Path(voices_dir)
-        self.backend = backend
-        self.upstream = upstream
-        self.upstream_model = upstream_model
-        self.llama_bin = llama_bin
-        self.llama_model = llama_model
-        self.llama_mmproj = llama_mmproj
-        self.llama_lang = llama_lang
-        self.llama_threads = llama_threads
-        self.port = port
-        self.max_chars = max_chars
-        self.timeout = timeout
-        self.default_voice = default_voice
-
-    @classmethod
-    def from_env(cls, environ=None):
-        env = os.environ if environ is None else environ
-        return cls(
-            voices_dir=env.get("VOICES_DIR", "/voices"),
-            backend=(env.get("TTS_BACKEND") or "omni-http").strip().lower(),
-            upstream=(env.get("TTS_UPSTREAM") or "http://host.containers.internal:8880").rstrip("/"),
-            upstream_model=env.get("TTS_UPSTREAM_MODEL") or "voxcpm2",
-            llama_bin=env.get("LLAMA_TTS_BIN") or "",
-            llama_model=env.get("LLAMA_TTS_MODEL") or "",
-            llama_mmproj=env.get("LLAMA_TTS_MMPROJ") or "",
-            llama_lang=env.get("LLAMA_TTS_LANG") or "english",
-            llama_threads=_int(env, "LLAMA_TTS_THREADS", 8),
-            port=_int(env, "SHIM_PORT", 7772),
-            max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
-            timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
-            default_voice=(env.get("DEFAULT_VOICE") or "").strip(),
-        )
-
-
-def _int(env, name, default):
-    try:
-        return int(str(env.get(name, "")).strip())
-    except (TypeError, ValueError):
-        return default
-
-
-def list_voices(voices_dir):
-    """Voice names in ``voices_dir``: directories that hold a ``reference.wav``.
-
-    A missing or unreadable directory is an empty fleet, not an error — the
-    health endpoint must still answer so the unit reports its state.
-    """
-    try:
-        entries = sorted(p.name for p in voices_dir.iterdir()
-                         if p.is_dir() and (p / "reference.wav").is_file())
-    except OSError:
-        return []
-    return entries
-
-
-def resolve_voice(voices_dir, voice):
-    """``$VOICES_DIR/<voice>/reference.wav`` for a valid name, else ``None``."""
-    if not isinstance(voice, str) or not VOICE_NAME_RE.fullmatch(voice):
-        return None
-    reference = voices_dir / voice / "reference.wav"
-    return reference if reference.is_file() else None
-
-
-def to_mp3(wav_bytes, timeout):
-    """Re-encode WAV bytes to mp3 with the image's ``ffmpeg`` (pipes, no temp
-    file: the audio never lands on disk to be left behind)."""
-    argv = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0",
-            "-f", "mp3", "pipe:1"]
-    try:
-        proc = subprocess.run(argv, input=wav_bytes, capture_output=True,
-                              timeout=timeout)
-    except FileNotFoundError:
-        raise ShimError(500, "response_format=mp3 needs the ffmpeg binary, which is not on PATH")
-    except subprocess.TimeoutExpired:
-        raise ShimError(502, "ffmpeg mp3 conversion timed out after %gs" % timeout)
-    if proc.returncode != 0 or not proc.stdout:
-        raise ShimError(502, "ffmpeg mp3 conversion failed (exit %s): %s"
-                        % (proc.returncode, _tail(proc.stderr)))
-    return proc.stdout
-
-
-def _tail(raw):
-    """Last line of a subprocess' stderr, for an error message. Text only —
-    audio bytes are never echoed, and never logged."""
-    text = (raw or b"").decode("utf-8", "replace").strip()
-    return text.splitlines()[-1][:300] if text else "no output"
-
-
-def _error(message, voices=None):
-    body = {"error": {"message": message, "type": "invalid_request_error"}}
-    if voices is not None:
-        body["error"]["available_voices"] = voices
-    return body
-
-
-class SpeechHandler(BaseHTTPRequestHandler):
-    protocol_version = "HTTP/1.1"
-    server_version = "frank-tts"
-    sys_version = ""
-
-    # -- plumbing ----------------------------------------------------------
-    def config(self):
-        return getattr(self.server, "config", None) or Config.from_env()
-
-    def log_message(self, fmt, *args):
-        # One prefix for the journal. Request lines and outcomes only: a
-        # synthesized text, a base64 reference voice and the audio bytes
-        # itself never reach stderr, in success or in failure.
-        sys.stderr.write("frank-tts: %s - %s\n"
-                         % (self.address_string(), fmt % args))
-
-    def log_error(self, fmt, *args):
-        # The base class routes refused requests through a separate sink; keep
-        # them on the same line format so the journal reads uniformly.
-        self.log_message(fmt, *args)
-
-    def _json(self, status, obj, head=False):
-        self._bytes(status, json.dumps(obj).encode("utf-8"), "application/json", head)
-
-    def _bytes(self, status, data, content_type, head=False):
-        self.send_response(status)
-        self.send_header("Content-Type", content_type)
-        self.send_header("Content-Length", str(len(data)))
-        self.end_headers()
-        if not head:
-            self.wfile.write(data)
-
-    def _drain(self):
-        """Read and discard a body this route will not answer. HTTP/1.1
-        keep-alive makes an unread body the NEXT request line, so a client
-        that reuses the connection would see a parse error it never sent."""
-        if getattr(self, "_body_read", False):
-            return  # already consumed; a second read would block the response
-        try:
-            length = int(self.headers.get("Content-Length") or 0)
-        except ValueError:
-            return
-        if length > 0:
-            self.rfile.read(length)
-
-    def _body(self):
-        """Parse the JSON request body. Never returned to a log or an error."""
-        try:
-            length = int(self.headers.get("Content-Length") or 0)
-        except ValueError:
-            raise ValueError("Content-Length is not a number")
-        raw = self.rfile.read(length) if length > 0 else b""
-        self._body_read = True
-        if not raw:
-            raise ValueError("expected a JSON body")
-        parsed = json.loads(raw.decode("utf-8"))
-        if not isinstance(parsed, dict):
-            raise ValueError("expected a JSON object")
-        return parsed
-
-    # -- routes ------------------------------------------------------------
-    def do_GET(self, head=False):
-        # HEAD shares the route table: `wget --spider` (the fleet's probe
-        # shape) asks for HEAD, and a 501 there would read as a dead service.
-        path = urllib.parse.urlparse(self.path).path
-        cfg = self.config()
-        if path == "/health":
-            self._json(200, {"status": "ok", "backend": cfg.backend,
-                             "voices": list_voices(cfg.voices_dir)}, head)
-        elif path == "/v1/audio/voices":
-            self._json(200, list_voices(cfg.voices_dir), head)
-        else:
-            self._json(404, _error("no such endpoint: %s" % path), head)
-
-    def do_HEAD(self):
-        self.do_GET(head=True)
-
-    def do_POST(self):
-        path = urllib.parse.urlparse(self.path).path
-        if path != "/v1/audio/speech":
-            self._drain()
-            self._json(404, _error("no such endpoint: %s" % path))
-            return
-        cfg = self.config()
-        try:
-            body = self._body()
-        except (ValueError, UnicodeDecodeError) as exc:
-            self._drain()
-            self._json(400, _error("malformed request body: %s" % exc))
-            return
-
-        text = body.get("input")
-        if not isinstance(text, str) or not text.strip():
-            self._json(400, _error("`input` must be a non-empty string"))
-            return
-        if len(text) > cfg.max_chars:
-            self._json(414, _error("input is %d characters, over the %d limit "
-                                   "(SHIM_MAX_CHARS); split it or raise the limit"
-                                   % (len(text), cfg.max_chars)))
-            return
-
-        fmt = str(body.get("response_format") or "wav").strip().lower()
-        if fmt not in ("wav", "mp3"):
-            self._json(400, _error("unsupported response_format %r; this "
-                                   "endpoint serves wav and mp3" % fmt))
-            return
-
-        # `voice_id` is the alias the omp harness forwards for `voice`. A request
-        # that names nothing takes the declared default — the harness has no
-        # setting that feeds a voice, so voice-less calls are the normal shape
-        # there, not a client bug.
-        requested = body.get("voice") or body.get("voice_id")
-        defaulted = not requested and bool(cfg.default_voice)
-        voice = requested or cfg.default_voice
-        reference = resolve_voice(cfg.voices_dir, voice)
-        if reference is None:
-            available = list_voices(cfg.voices_dir)
-            listed = ", ".join(available) if available else "(none: no voice pack with a reference.wav in %s)" % cfg.voices_dir
-            self._json(400, _error("unknown voice %r; available voices: %s"
-                                   % (voice, listed), available))
-            return
-
-        model = str(body.get("model") or cfg.upstream_model)
-        try:
-            if cfg.backend == "omni-http":
-                audio = self._speech_http(cfg, model, text, str(voice), reference)
-            elif cfg.backend == "llama-tts-cli":
-                audio = self._speech_cli(cfg, text, reference)
-            else:
-                raise ShimError(500, "unknown TTS_BACKEND %r (expected omni-http or llama-tts-cli)" % cfg.backend)
-            if fmt == "mp3":
-                audio = to_mp3(audio, cfg.timeout)
-        except ShimError as exc:
-            self._json(exc.status, _error(exc.message))
-            return
-
-        content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
-        self._bytes(200, audio, content_type)
-        self.log_message("speech voice=%s%s format=%s chars=%d -> %d bytes %s",
-                         voice, " (default)" if defaulted else "", fmt,
-                         len(text), len(audio), content_type)
-
-    # -- backends ----------------------------------------------------------
-    def _speech_http(self, cfg, model, text, voice, reference):
-        """Forward to the upstream with the cloned voice riding along as
-        base64 ``reference_audio``. The upstream is asked for wav always; mp3
-        is our own re-encode on the way out."""
-        try:
-            payload = {
-                "model": model,
-                "input": text,
-                "voice": voice,
-                "response_format": "wav",
-                "reference_audio": base64.b64encode(reference.read_bytes()).decode("ascii"),
-            }
-        except OSError as exc:
-            raise ShimError(500, "cannot read %s: %s" % (reference, exc))
-
-        url = cfg.upstream + "/v1/audio/speech"
-        request = urllib.request.Request(
-            url, data=json.dumps(payload).encode("utf-8"),
-            headers={"Content-Type": "application/json"}, method="POST")
-        try:
-            with urllib.request.urlopen(request, timeout=cfg.timeout) as response:
-                audio = response.read()
-        except urllib.error.HTTPError as exc:
-            detail = exc.read(512).decode("utf-8", "replace").strip()
-            raise ShimError(502, "upstream %s returned HTTP %d: %s"
-                            % (url, exc.code, detail or "no detail"))
-        except urllib.error.URLError as exc:
-            raise ShimError(502, "upstream %s unreachable: %s" % (url, exc.reason))
-        except (TimeoutError, OSError) as exc:
-            raise ShimError(502, "upstream %s failed: %s" % (url, exc))
-        if not audio:
-            raise ShimError(502, "upstream %s returned no audio" % url)
-        return audio
-
-    def _speech_cli(self, cfg, text, reference):
-        """Run the local TTS binary once, in a temp dir that is always removed
-        — also on failure, or a crash-looping binary fills /tmp."""
-        if not cfg.llama_bin or not cfg.llama_model:
-            raise ShimError(500, "TTS_BACKEND=llama-tts-cli needs LLAMA_TTS_BIN and LLAMA_TTS_MODEL")
-        work = tempfile.TemporaryDirectory(prefix="frank-tts-")
-        try:
-            out_path = Path(work.name) / "speech.wav"
-            argv = ["nice", "-n", "19", "env", "OMP_NUM_THREADS=%d" % cfg.llama_threads,
-                    cfg.llama_bin, "-m", cfg.llama_model]
-            if cfg.llama_mmproj:
-                argv += ["-mm", cfg.llama_mmproj]
-            argv += ["--tts-lang", cfg.llama_lang,
-                     "--tts-speaker-file", str(reference),
-                     "-p", text,
-                     "--output", str(out_path)]
-            try:
-                proc = subprocess.run(argv, cwd=work.name, capture_output=True,
-                                      timeout=cfg.timeout)
-            except FileNotFoundError as exc:
-                raise ShimError(500, "cannot run the TTS command: %s" % exc)
-            except subprocess.TimeoutExpired:
-                raise ShimError(502, "TTS binary timed out after %gs" % cfg.timeout)
-            if proc.returncode != 0:
-                raise ShimError(502, "TTS binary failed (exit %s): %s"
-                                % (proc.returncode, _tail(proc.stderr)))
-            if not out_path.is_file():
-                raise ShimError(502, "TTS binary exited 0 but wrote no %s" % out_path.name)
-            return out_path.read_bytes()
-        finally:
-            work.cleanup()
-
-
-class SpeechServer(ThreadingHTTPServer):
-    """Threads so a health probe answers while a long synthesis runs; daemon
-    threads so SIGTERM is never held hostage by a stuck request."""
-
-    daemon_threads = True
-    allow_reuse_address = True
-    config = None
-
-
-def create_server(config=None, host="0.0.0.0", port=None):
-    """Bind the shim. ``config``/``port`` are the injectable seams: tests pass
-    a Config with a temp VOICES_DIR and ``port=0`` for an ephemeral port."""
-    cfg = Config.from_env() if config is None else config
-    server = SpeechServer((host, cfg.port if port is None else port), SpeechHandler)
-    server.config = cfg
-    return server
-
-
-def main(argv=None):
-    parser = argparse.ArgumentParser(
-        prog="frank-tts", description="OpenAI-compatible speech endpoint for cloned voice packs")
-    parser.add_argument("--host", default="0.0.0.0",
-                        help="bind address (default: all interfaces)")
-    parser.add_argument("--port", type=int, default=None,
-                        help="bind port (default: $SHIM_PORT, 0 = ephemeral)")
-    args = parser.parse_args(argv)
-
-    server = create_server(host=args.host, port=args.port)
-    cfg = server.config
-    bound = server.server_address[1]
-    print("frank-tts: backend=%s voices_dir=%s listening on http://%s:%d"
-          % (cfg.backend, cfg.voices_dir, args.host, bound), flush=True)
-    try:
-        server.serve_forever()
-    except KeyboardInterrupt:
-        pass
-    finally:
-        server.server_close()
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())
@@ -1,134 +0,0 @@
-# Quadlet unit: frank-tts-engine (rootless, boot-persistent).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.container
-# Unit name: frank-tts-engine.service (container name: frank-tts-engine)
-#
-# WHAT IT IS. The arm A TTS engine: qwentts.cpp (pinned commit
-# 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4) built into an image by
-# dankclaw/voiceservice/Containerfile.engine, running its resident `tts-server`
-# with the 1.7 B base talker Q4_K_M and the Q8_0 12 Hz tokenizer loaded ONCE at
-# startup. It is deliberately NOT a fleet endpoint: the endpoint consumers
-# address is **frank-tts on 7772** ([[Frank TTS]]), which resolves voice names,
-# re-encodes to mp3 and owns the timeout budget. Runbook: [[Frank TTS Engine]].
-#
-# WHY THIS ENGINE (Frank's call, 2026-10-18, criterion "whatever is better for
-# live output"): server-side time-to-first-audio 94-112 ms (86-451 ms end to
-# end through the façade), streaming that stays ahead of real time at warm RTF
-# 0.55-0.67, and a run-to-run spread of +-0.01 — against the mainline CLI path's
-# 0.93-0.99 at +-0.25, which crosses 1.0 often enough to stall a live stream.
-# The A/B that decided it is "Decision: arm A for live output" in
-# [[Local TTS and Voice Cloning]].
-#
-# IMAGE. Built locally by deploy-remote.sh (build_compose_image), like
-# loki/alloy/frank-tts. `Pull=never` plus the `localhost/` prefix are required:
-# this host's quadlet path cannot resolve a locally built tag from local storage
-# otherwise. The build is a network operation (it clones the pinned commit) and
-# is skipped when the Containerfile hash label already matches.
-#
-# NETWORK, AND WHY IT PUBLISHES NOTHING. The container sits on exactly one
-# network: the private `frank-tts-engine.network` (10.89.5.0/24). The façade
-# joins the same network and dials `http://frank-tts-engine:8880` — the
-# container DNS name, on the network that owns it. There is no `PublishPort` of
-# any kind, not even `127.0.0.1:8880`, and the reason is the lesson that 502'd
-# hermes: netavark DNATs published ports only, so a host-loopback publish is not
-# a path to a peer container — the façade could never have used it, and a bridge
-# container cannot reach the host's own loopback bind either (see
-# [[Container Network Namespaces]]). A publish would therefore buy exactly one
-# thing: an extra reachable surface on an engine that has no authentication, on
-# a box whose CPU is also carrying the text endpoints. Diagnostics do not need
-# it either — `podman exec frank-tts-engine curl http://127.0.0.1:8880/health`
-# speaks to the server from inside its own namespace, which is where the address
-# is valid.
-#
-# MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs:
-# the Vulkan backend copies both tensors into device memory at load, so they
-# stop counting in the process's RSS (measured after the 2026-10-19 cutover —
-# `podman stats` / `ps -o rss`). No memory limit is set: the box has the room
-# and a limit would convert a busy moment into an OOM kill mid-utterance.
-#
-# `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan
-# ICD enumerates nothing and the server dies on device lookup after it has
-# already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the
-# thing that works here: the units podman spawns carry no supplementary groups
-# at all, and this host grants render access a different way — the udev rule
-# that makes the render nodes world-writable (same rule ComfyUI rides, see
-# dankclaw/comfyui). Measured: `id` inside this container lists only the
-# container's own group, and `tts-server` still holds renderD129.
-# The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and
-# renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a
-# re-probe can invalidate, while the directory hands the container every render
-# node and the driver picks by name.
-#
-# /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
-# the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
-# container a second way onto the GPU is surface for nothing.
-#
-# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
-# becomes known to the engine only through POST /v1/audio/voices, and the
-# registration is MEMORY ONLY — it does not survive a restart, which is exactly
-# why the warm hook below runs on every start. That hook runs INSIDE this
-# container (see ExecStartPost), so it reads the packs at /voices; `tts-server`
-# itself never opens the directory, it receives the reference audio as base64 in
-# the registration body. The façade mounts the same host directory read-only for
-# its own name list, so one directory stays the single source of a voice name.
-#
-# REGISTRY LAW, THE REASON EXECSTARTPOST EXISTS. The voice registry is memory
-# only: a restart forgets every voice, and re-registering does not warm one —
-# only synthesizing does — so a boot without this hook leaves the façade
-# answering 400 for every name. The per-voice cold penalty is real but small, and
-# not the trial's 6.2-8.0 s: measured on the built container 2026-10-18 against an
-# empty registry, the first synthesis of a freshly registered voice cost 1,883 ms
-# against 1,226-1,283 ms for the requests after it. The warm pass runs serially
-# (two short syntheses per voice, ~5 s each) and `TimeoutStartSec=900` keeps a growing library from
-# tripping systemd's default 90 s start deadline. The leading `-` on
-# ExecStartPost is load-bearing: with `Restart=always`, a warmer that exits
-# non-zero (a pack missing its reference.wav, the engine still loading past the
-# warmer's own patience) would restart the whole container and crash-loop the
-# endpoint forever. A failed warm must be a journal line, not a restart.
-#
-# The hook is `podman exec`, not a host script, because there is no address the
-# host could dial: this container publishes nothing (above). Exec-ing into it
-# also means the warmer inherits this unit's EnvironmentFile, so every knob it
-# has lives in scripts/frank-tts-engine.env beside the engine's own, and it sees
-# the same /voices and 127.0.0.1:8880 the engine does.
-#
-# HEALTH. `GET /health` answers `{"status":"ok"}` from the HTTP thread, not the
-# synthesis queue, so a busy engine still reads healthy and a dead one does not.
-# HealthStartPeriod covers the first model load. The command is argument-only
-# and needs /bin/sh — the generator wraps HealthCmd as CMD-SHELL — so the image
-# installs curl (see the opposite case in dankclaw/loki, whose distroless base
-# has no shell).
-
-[Unit]
-Description=frank-tts-engine (arm A qwentts.cpp TTS engine, internal :8880)
-After=network-online.target
-Wants=network-online.target
-
-[Container]
-ContainerName=frank-tts-engine
-Image=localhost/franksdank/frank-tts-engine:latest
-Pull=never
-Network=frank-tts-engine.network
-# No PublishPort, at all: the only reachable address this server has is the
-# bridge address the façade resolves by name. Diagnostics use `podman exec`.
-EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
-Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
-Volume=/home/dank/voices:/voices:ro
-# Render access for the iGPU (see MEMORY / THE RENDER NODE above).
-AddDevice=/dev/dri
-GroupAdd=keep-groups
-HealthCmd=curl -fsS http://127.0.0.1:8880/health
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=120s
-
-[Service]
-Restart=always
-RestartSec=10
-# Model load (tens of seconds) plus two short syntheses per registered voice.
-TimeoutStartSec=900
-ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
-
-[Install]
-WantedBy=default.target
@@ -1,52 +0,0 @@
-# Quadlet network: the private bridge of the voice pair (rootless).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.network
-# Unit name: frank-tts-engine-network.service (podman network name:
-# systemd-frank-tts-engine — quadlet prefixes "systemd-", so probe the runtime
-# name, never the filename; see [[Container Network Namespaces]]).
-#
-# WHY A PRIVATE BRIDGE FOR A PAUSE THAT LOOKS PUBLIC-FACING. frank-tts (:7772,
-# published, the one endpoint consumers address) and frank-tts-engine (:8880,
-# never published, the arm A qwentts.cpp server behind it) could both have
-# stayed on the fleet-wide dankclaw_default and resolved each other by name.
-# They are split off anyway, for two reasons:
-#
-# 1. THE HOST-LOOPBACK TRAP. The phase-1 shape in `scripts/frank-tts.env` still
-#    names `host.containers.internal:8880` as the upstream, on the assumption the
-#    engine runs on the host and publishes `127.0.0.1:8880`. That cannot work:
-#    netavark DNATs published ports only, a published port is not on any podman
-#    bridge, and a container cannot reach the host's own loopback bind — the same
-#    shape that 502'd hermes, and the reason the signal-relay hop needs a host LAN
-#    bind instead ([[Container Network Namespaces]], [[Homepage Status Probes]]).
-#    Putting both containers on one committed network removes the guess: the
-#    façade dials the engine's container DNS name on the network that owns it, and
-#    the engine publishes nothing at all.
-#
-# 2. THE ENGINE HAS NO AUTH AND NO QUEUE LIMIT WORTH THE NAME. `tts-server`
-#    answers anyone who can open a socket, and at `--max-batch 1` a second
-#    caller simply waits behind the first. Left on dankclaw_default every
-#    container on that net — open-webui, hermes, the *arr proxies — could drive
-#    a CPU synthesiser that occupies every other core on a box that also serves
-#    two text endpoints. On this bridge the only dialer is frank-tts: the engine
-#    publishes no port, so not even the host has an address to reach it at, and
-#    diagnostics go through `podman exec` instead.
-#
-# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
-# per-container userspace helper left a container that accepted TCP and reset
-# every request for two days, logging nothing). A bridge has no such helper.
-#
-# Subnet registry on dankclaw (rootless):
-#   10.88.0.0/16  default "podman" bridge
-#   10.89.0.0/24  gitea (systemd-gitea)
-#   10.89.1.0/24  compose stack (dankclaw_default)
-#   10.89.2.0/24  searxng (systemd-searxng)
-#   10.89.3.0/24  open-webui (systemd-open-webui)
-#   10.89.4.0/24  open-notebook (systemd-open-notebook)
-#   10.89.5.0/24  voice pair (systemd-frank-tts-engine)  <- this file
-# Pinned so the two container IPs stay deterministic across reboots, like the
-# other pinned nets; 10.89.5.0/24 was the next unclaimed /24 in this registry.
-
-[Network]
-Driver=bridge
-Subnet=10.89.5.0/24
-Gateway=10.89.5.1
@@ -1,84 +0,0 @@
-# Quadlet unit: frank-tts (rootless, boot-persistent).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/frank-tts.container
-# Unit name: frank-tts.service (container name: frank-tts)
-#
-# WHAT IT IS. The fleet's one OpenAI-compatible speech endpoint: consumers
-# (omp, OpenWebUI, open-notebook) POST /v1/audio/speech with voice=<name> and
-# get audio in that cloned voice. A voice is a directory under
-# /home/dank/voices holding reference.wav; the shim refuses any name without
-# one (400 + the list that does exist) rather than narrating in a fallback
-# voice nobody chose. The engine behind it is a separate decision —
-# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to the
-# OpenAI-shaped engine (`frank-tts-engine`, [[Frank TTS Engine]]) and running a
-# local CLI binary per request.
-#
-# IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
-# deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
-# pinned by digest + ffmpeg, app.py copied in, running as uid 10000.
-# Pull=never and the localhost/ prefix are required — this host's quadlet
-# path cannot resolve a locally built tag from local storage otherwise.
-#
-# NETWORK. TWO attached networks, and the order is the one the open-notebook
-# pair uses: `dankclaw_default` first (the default route, and where consumers
-# live — they reach the published 7772, and open-notebook dials
-# `http://frank-tts:7772/v1` by container name), then the voice pair's private
-# bridge, `frank-tts-engine.network`, where the arm A engine lives. The shim's
-# upstream is therefore the engine's container DNS name
-# (`http://frank-tts-engine:8880`, scripts/frank-tts.env) and NOT
-# host.containers.internal: netavark DNATs published ports only, so a bridge
-# container cannot reach a host-loopback bind — the shape that 502'd hermes —
-# and the engine publishes nothing at all. That file MUST be installed beside
-# this unit (the engine's handler does it; the façade handler re-installs it for
-# `--only frank-tts`), because with a referenced Network= file missing the v1
-# generator refuses to produce this container unit at all.
-#
-# ORDERING. Wants+After on frank-tts-engine.service, not Requires: the façade
-# has a second backend (`TTS_BACKEND=llama-tts-cli`) that needs no upstream, and
-# an engine that will not start must not take this endpoint's unit down with it.
-# When both boot, this comes up after the engine, whose ExecStartPost warm pass
-# is part of its start — so the first request of the day meets a REGISTERED voice
-# (the registry is memory-only) rather than a 400, and skips the ~0.6 s penalty
-# the first synthesis of an unwarmed voice costs.
-#
-# VOICES. /home/dank/voices is host state, never repo state, and is mounted
-# READ-ONLY: this endpoint has no business writing voice packs. Because the
-# container runs as uid 10000, the packs must be world-readable (644 files,
-# 755 directories), which is how the pack builder writes them.
-#
-# NO GPU. The shim is a façade: audio conversion via ffmpeg on the CPU and,
-# for the CLI backend, whatever CPU threads LLAMA_TTS_THREADS allows, at nice
-# 19. No /dev/kfd, no /dev/dri, no keep-groups — unlike comfyui, whose unit
-# carries them deliberately.
-[Unit]
-Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
-After=network-online.target frank-tts-engine.service
-Wants=network-online.target
-Wants=frank-tts-engine.service
-
-[Container]
-ContainerName=frank-tts
-Image=localhost/franksdank/frank-tts:latest
-Pull=never
-Network=dankclaw_default
-Network=frank-tts-engine.network
-PublishPort=7772:7772
-EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
-Volume=/home/dank/voices:/voices:ro
-# The probe is a probe, not a voice test: /health answers 200 with the backend
-# and the resolved voice list, so a mounted-but-empty /voices still reads
-# healthy while a dead app does not. busybox wget ships in the alpine base;
-# keep the command argument-only — the podman 5.x generator splits HealthCmd
-# on spaces (see the comfyui unit for the quoting trap).
-HealthCmd=wget --spider -q http://127.0.0.1:7772/health
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=30s
-
-[Service]
-Restart=always
-RestartSec=10
-
-[Install]
-WantedBy=default.target
@@ -1,124 +0,0 @@
-#!/usr/bin/env bash
-# tts-engine-start.sh — the entrypoint of the frank-tts-engine image
-# (dankclaw/voiceservice/Containerfile.engine). It exists for one reason: the
-# knobs of the arm A engine are command-line flags, while the fleet's knobs
-# live in scripts/frank-tts-engine.env and reach the container through
-# EnvironmentFile=. This is the translation layer, and nothing else — no
-# supervision, no retries, no logging of its own. PID 1 stays the server, so
-# `podman stop` and the unit's Restart=always drive it directly.
-#
-# THE FLAG THAT MUST STAY ABSENT:
-#
-#   --threads / -t   does not exist in this build. src/backend.h computes the
-#                    CPU thread count as std::thread::hardware_concurrency()/2
-#                    (the comment there: GEMM shares SIMD units across
-#                    hyperthreads), and there is no flag to override it. So a
-#                    CPU affinity pin (CPUQuota=, taskset, a cpuset) silently
-#                    changes the engine's thread count instead of just its
-#                    placement — pinning is deliberately not done anywhere in
-#                    this stack. See the env file header for the same warning
-#                    aimed at the operator.
-#
-# --device IS here, but conditionally: it reaches the binary only when
-# QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
-# takes its own default device, which is CPU because the CPU backend is always
-# built. One line in scripts/frank-tts-engine.env moves the endpoint between the
-# two shapes, with no rebuild. What must never reach argv is a device NAME the
-# loader does not enumerate: that is not refused by the argument parser, it dies
-# during device lookup with the loader's own error, so the journal reads as a
-# load that failed rather than a configuration refusal.
-#
-# Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
-# the Q8_0 tokenizer, one request at a time, on the private bridge, on the
-# gfx1151 iGPU. An empty environment still boots that shape.
-set -euo pipefail
-
-# The binary path. Overridable for one reason: the bats seam runs a stub
-# instead of a 1.2 GiB model. Operators never set it — the image ships exactly
-# one binary at /app/tts-server.
-: "${QWENTTS_BIN:=/app/tts-server}"
-
-# Paths are CONTAINER paths (the host directories they come from are named in
-# scripts/frank-tts-engine.env and mounted by the Quadlet unit).
-: "${QWENTTS_MODEL:=/models/qwen-talker-1.7b-base-Q4_K_M.gguf}"
-: "${QWENTTS_CODEC:=/models/qwen-tokenizer-12hz-Q8_0.gguf}"
-
-# Reported by GET /v1/models, and what scripts/frank-tts.env's
-# TTS_UPSTREAM_MODEL names. A model id, not a filename: the file name would
-# leak a quantization choice into every consumer's config.
-: "${QWENTTS_ALIAS:=frank-tts-engine}"
-
-# 0.0.0.0 inside the container is not "public". tts-server defaults to
-# 127.0.0.1 (its own usage prints `--host <ip> Listen address (default:
-# 127.0.0.1)`), and inside a container netns a loopback bind is reachable by
-# nothing outside that namespace — the façade dials this container by its bridge
-# address on the private voice network, which is why the address has to be
-# broad. The unit PUBLISHES NOTHING (see
-# dankclaw/voiceservice/quadlet/frank-tts-engine.container: no PublishPort at
-# all, not even 127.0.0.1:8880), so the only namespaces that can reach
-# 0.0.0.0:8880 are that bridge and this container's own loopback — and its own
-# loopback is exactly where the boot warmer and the health probe speak.
-: "${QWENTTS_HOST:=0.0.0.0}"
-: "${QWENTTS_PORT:=8880}"
-
-# The language label used when a request omits one. `auto` is the upstream
-# default and what the trial ran.
-: "${QWENTTS_LANG:=auto}"
-
-# One request at a time. The server-side queue is a serialised FIFO at this
-# setting, which is what keeps the per-voice timings predictable. A second
-# concurrent synthesis shares the device with the first — and on this box that
-# device is also carrying frank-dragoon — so it buys no throughput and it
-# doubles the wait for the caller behind it. The engine's own default is also 1;
-# written here because it is a decision, not an accident.
-: "${QWENTTS_MAX_BATCH:=1}"
-
-# The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
-# the gfx1151 iGPU on this host. An index would follow whatever order the
-# loader's device list returns; a name says which device this endpoint means.
-# The container sees exactly one Vulkan device because the image carries the
-# Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
-#
-# EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
-# the binary picks its own default, and the CPU backend is always built — which
-# is why moving this endpoint between the iGPU and the CPU is one line in
-# scripts/frank-tts-engine.env and not a rebuild.
-#
-# That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
-# fires on a set-but-empty variable as well as an unset one, so the rollback line
-# `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
-# on the shared GPU with no way out short of a rebuild. Only UNSET takes the
-# default.
-if [[ ! -v QWENTTS_DEVICE ]]; then
-    QWENTTS_DEVICE=Vulkan0
-fi
-
-# Fail loudly BEFORE binding, so the journal says which artifact is missing
-# instead of the unit reporting a container that starts and dies. This is the
-# self-healing path: Restart=always re-runs it every RestartSec, and the same
-# sentence repeats until the operator fetches the weights
-# (scripts/download-frank-tts-engine-qwentts.sh).
-for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
-    if [[ ! -r "$artifact" ]]; then
-        echo "frank-tts-engine: cannot read ${artifact}; run scripts/download-frank-tts-engine-qwentts.sh on the host (the /models mount is read-only and holds the two GGUF files)" >&2
-        exit 1
-    fi
-done
-
-ARGS=(
-    --model "$QWENTTS_MODEL"
-    --codec "$QWENTTS_CODEC"
-    --alias "$QWENTTS_ALIAS"
-    --host "$QWENTTS_HOST"
-    --port "$QWENTTS_PORT"
-    --lang "$QWENTTS_LANG"
-    --max-batch "$QWENTTS_MAX_BATCH"
-)
-
-# Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
-# because "the rollback shape" is argv identical to the pre-iGPU endpoint.
-if [[ -n "$QWENTTS_DEVICE" ]]; then
-    ARGS+=(--device "$QWENTTS_DEVICE")
-fi
-
-exec "$QWENTTS_BIN" "${ARGS[@]}"
@@ -1,222 +0,0 @@
-#!/usr/bin/env bash
-# tts-engine-warm.sh — the boot warmer of the arm A TTS engine, shipped inside
-# the frank-tts-engine image at /app/warm-voices.sh
-# (dankclaw/voiceservice/Containerfile.engine).
-#
-# WHO RUNS IT. Two callers, same binary:
-#
-#   1. The Quadlet unit, as
-#      `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
-#      — after every start of the container, including every boot.
-#      `podman exec` rather than a host script because the engine publishes no
-#      port at all, so there is no address the host could dial; and because an
-#      exec inherits the unit's EnvironmentFile, which is what puts every knob of
-#      this script in scripts/frank-tts-engine.env next to the engine's own.
-#   2. scripts/deploy-remote.sh's deploy_frank_tts_engine, once per deploy, so a
-#      voice that did not warm becomes a deploy result instead of only a journal
-#      line (the unit's `-` prefix deliberately cannot fail the unit).
-#
-# WHY IT EXISTS AT ALL (the registry law). The voice registry is MEMORY ONLY: a
-# restart forgets every voice, and re-registering a voice does not warm it — only
-# synthesizing does. A boot that registers nothing therefore leaves frank-tts
-# answering 400 for every name, so this script registers every pack it finds. The
-# per-voice cold penalty is real but small, and smaller than the trial reported:
-# measured on the deployed build 2026-10-18 against an empty registry, the FIRST
-# synthesis of a freshly registered voice cost 1,883 ms against 1,226-1,283 ms for
-# the requests after it — about 0.6 s, not the trial's 6.2-8.0 s. Each voice gets
-# two identical requests so the journal carries both numbers, `voice <name>
-# cold=…ms warm=…ms`; cold ≈ warm is the expected shape, and a large gap is what
-# would mean the snapshot path is broken.
-#
-# THE VOICES. /voices — the host directory /home/dank/voices mounted read-only by
-# the unit, the same packs frank-tts mounts for its own name list. A pack is a
-# directory that contains reference.wav; everything else in it (manifest.json, the
-# engine's reference.spk / reference.rvq extraction, a 16 kHz variant) is not read
-# here. The registration payload is `{"name": <dir>, "wav_b64": <base64 of
-# reference.wav>}` with NO ref_text, which selects the embedding-only (x-vector)
-# voice — the shipped default, warm RTF ~0.57-0.63 against ICL's 0.67, and the one
-# this fleet can produce honestly because it has no trustworthy transcript for the
-# audition crops. Adding ref_text (with the codes) would flip that voice to
-# in-context-learning mode: better fidelity on a good transcript, slower, and only
-# worth it for batch work like a podcast render. That is a per-voice opt-in this
-# script deliberately does NOT exercise, and the door stays open: a pack that
-# wants ICL needs its text in the pack plus one key in the payload below.
-#
-# A lone `spk_b64` is a 400 (src/tts-server.h:480 — it is `wav_b64` or BOTH
-# `spk_b64` and `rvq_b64`), which is why the WAV travels and the .spk stays home.
-#
-# NOTHING HERE IS A SECRET, and nothing is logged that could be: the audio bytes
-# are discarded, the synthesized text comes from the env file, and the reference
-# clip is base64'd into a request body that is written to a temporary file and
-# handed to curl as `@file` — never echoed, and never placed on a command line,
-# because of the size law explained at the mktemp below.
-set -euo pipefail
-
-# The engine's own address, from inside its namespace. Loopback is valid HERE:
-# this script runs in the container, next to the server. There is deliberately no
-# host-visible address to use instead — the unit publishes nothing, because a
-# host-loopback publish would be unreachable from the façade anyway (netavark
-# DNATs published ports only) while adding a surface on an unauthenticated
-# server.
-: "${ENGINE_URL:=http://127.0.0.1:8880}"
-
-# Container path; the host directory behind it is named in the Quadlet unit.
-: "${VOICES_DIR:=/voices}"
-
-# How long to wait for the engine to answer /health. Model load dominates: the
-# two GGUFs (1.2 GiB + 291 MB) are read and the context built before the HTTP
-# thread binds, and a boot races every other unit on this box.
-: "${HEALTH_WAIT:=300}"
-
-# One short sentence, long enough to exercise the codec snapshot and short enough
-# that warming a library of voices is seconds, not minutes. Never varied per run:
-# the cold/warm numbers in the journal should mean the same thing every boot. The
-# synthesis asks for `wav`, the buffered path — the streamed path (`pcm`) is
-# different audio at the same seed, and `wav` is also what frank-tts asks for.
-: "${WARM_TEXT:=Warm up, one two three.}"
-
-# Per-request ceiling. A cold voice is 6-8 s by measurement; anything approaching
-# this is a stalled engine and should be reported rather than waited on forever —
-# systemd counts the whole warm pass against the unit's start deadline.
-: "${REQUEST_TIMEOUT:=120}"
-
-# The warm text goes into a JSON string literal by interpolation. Two steps make
-# whatever the two readers of scripts/frank-tts-engine.env deliver arrive as the
-# sentence itself:
-#
-# 1. Unwrap ONE shell-style quoting layer. bash (`set -a; source`) strips the
-#    quotes around a value with spaces; podman's `--env-file` does NOT — measured
-#    2026-10-18 against the image's pinned base, `WARM_TEXT="…"` reaches the
-#    container WITH its quotes. Unwrapped only when the whole value is exactly one
-#    quoted string (starts AND ends with `"` and holds no quote inside), so a
-#    sentence that legitimately contains quotes is never mangled.
-# 2. Escape the two characters that would corrupt or end a JSON string: a
-#    backslash, and a double quote.
-#
-# Without step 1 the body is `{"input":""Warm up…"",…}`, the engine's JSON parser
-# rejects it, every voice stays cold, and the boot still reads healthy. Voice NAMES
-# need neither step: the character class checked below admits no quote, no
-# backslash and no control byte.
-warm_text="$WARM_TEXT"
-if (( ${#warm_text} >= 2 )) && [[ "$warm_text" == \"*\" ]]; then
-    inner="${warm_text:1:${#warm_text}-2}"
-    [[ "$inner" != *'"'* ]] && warm_text="$inner"
-fi
-warm_json="${warm_text//\\/\\\\}"
-warm_json="${warm_json//\"/\\\"}"
-
-# The registration body is a FILE, never an argument. The kernel caps a single
-# command-line string at 32 pages — MAX_ARG_STRLEN = 131,072 bytes — and a shipped
-# reference clip base64's to about 1.27 MB, so passing the body as one argument
-# fails with E2BIG before curl is even executed. The first real boot of the unit
-# (2026-10-18, the three shipped packs) proved it end to end: `/usr/bin/curl:
-# Argument list too long` for every voice, reported as the misleading
-# "registration failed (HTTP no response)". `mktemp` lands in the container's
-# /tmp as the unprivileged `tts` user; one file for the whole pass, overwritten
-# per voice, removed on any exit — it is container scratch, not host state.
-body_file="$(mktemp -t frank-tts-warm.XXXXXX)"
-trap 'rm -f "$body_file"' EXIT
-
-fail=0
-log() { printf 'frank-tts-engine-warm: %s\n' "$*"; }
-
-# Wait for /health. It answers from the HTTP thread, so a 200 means the server is
-# listening; it says nothing about voices, which is what the rest of this script
-# is for.
-ready=0
-for ((i = 1; i <= HEALTH_WAIT; i++)); do
-    if curl -fsS --max-time 5 "${ENGINE_URL}/health" >/dev/null 2>&1; then
-        ready=1
-        break
-    fi
-    sleep 1
-done
-if (( ready == 0 )); then
-    log "engine did not answer ${ENGINE_URL}/health within ${HEALTH_WAIT}s; nothing warmed"
-    exit 1
-fi
-
-if [[ ! -d "$VOICES_DIR" ]]; then
-    log "no voice directory at ${VOICES_DIR}; nothing to warm (frank-tts will answer 400 with an empty voice list until packs are built)"
-    exit 0
-fi
-
-warmed=0
-for dir in "$VOICES_DIR"/*/; do
-    [[ -d "$dir" ]] || continue
-    name="${dir%/}"
-    name="${name##*/}"
-    if [[ ! -f "${dir}reference.wav" ]]; then
-        continue          # not a pack; frank-tts refuses this name too
-    fi
-    # The engine lowercases every voice name (tts_voice_name) while frank-tts
-    # matches the directory name exactly, so a mixed-case pack would work by
-    # accident on one side and not the other. Refuse it loudly instead. Pack
-    # names are also JSON string content here, so the character class is what
-    # keeps a directory name from writing structure into the request body.
-    if [[ ! "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
-        log "voice '${name}': name must be lowercase [a-z0-9._-] (the engine lowercases names; the shim does not) — skipped"
-        fail=1
-        continue
-    fi
-
-    # The audio enters through a redirection rather than a command substitution:
-    # the file is several hundred KB, and both a shell variable and an argument of
-    # that size are what this script used to die on.
-    {
-        printf '{"name":"%s","wav_b64":"' "$name"
-        base64 -w0 "${dir}reference.wav"
-        printf '"}'
-    } > "$body_file"
-    if ! code="$(curl -sS --max-time "$REQUEST_TIMEOUT" -o /dev/null -w '%{http_code}' \
-            -H 'Content-Type: application/json' --data-binary "@${body_file}" \
-            "${ENGINE_URL}/v1/audio/voices")" || [[ "$code" != "200" ]]; then
-        log "voice '${name}': registration failed (HTTP ${code:-no response}) — the shim will answer 400 for this name"
-        fail=1
-        continue
-    fi
-
-    # The cold pass. Its output is discarded; the point is the codec snapshot it
-    # leaves inside the engine.
-    start=$(date +%s%3N)
-    if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
-            -H 'Content-Type: application/json' \
-            --data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
-            "${ENGINE_URL}/v1/audio/speech"; then
-        log "voice '${name}': registered, but the warm synthesis failed — first live request stays cold"
-        fail=1
-        continue
-    fi
-    cold=$(( $(date +%s%3N) - start ))
-
-    # The warm pass, same request. This is the number the fleet actually
-    # experiences, and it is what "the second request answers under a second" is
-    # measured against.
-    start=$(date +%s%3N)
-    if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
-            -H 'Content-Type: application/json' \
-            --data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
-            "${ENGINE_URL}/v1/audio/speech"; then
-        log "voice '${name}': warm synthesis failed after a cold one that worked"
-        fail=1
-        continue
-    fi
-    warm=$(( $(date +%s%3N) - start ))
-
-    log "voice ${name} cold=${cold}ms warm=${warm}ms"
-    warmed=$(( warmed + 1 ))
-done
-
-# Failure first, emptiness second, and not the other way round: a pass where
-# every voice failed also has `warmed == 0`, and reading that as "nothing to do,
-# success" is how an endpoint that answers 400 for every voice name gets deployed
-# on a Friday.
-if (( fail )); then
-    log "${warmed} voice(s) warm, at least one failed (see the lines above)"
-    exit 1
-fi
-if (( warmed == 0 )); then
-    log "no voice with a reference.wav under ${VOICES_DIR}; engine is up but speaks nothing"
-    exit 0
-fi
-log "${warmed} voice(s) registered and warm"
@@ -0,0 +1,156 @@
+#!/usr/bin/env python3
+"""HTTP front door for the frank-medium single-GPU replicas.
+
+Why two replicas instead of TP=2: one vLLM process spanning both R9700s needs
+RCCL collectives, and RCCL's kernels require PCIe atomics on every
+participating GPU. The second card sits behind chipset bridges advertising
+`AtomicOpsCap: Routing-`, which no software can enable (see the runbook), so
+no collective can launch on it. Two single-GPU replicas use both cards; this
+service keeps the fleet contract of one endpoint on :7770, streaming included.
+
+Routing decisions live in routing.py — spread single-turn requests, pin
+continuations to the replica whose prefix cache holds their context. This file
+only moves bytes: parse enough of the request to route it, forward it verbatim,
+stream the answer back, and report replica health on /health.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+from contextlib import suppress
+from aiohttp import ClientSession, ClientTimeout, web
+
+import routing
+
+PORT = int(os.environ.get("PORT", "7770"))
+HEALTH_TTL = float(os.environ.get("HEALTH_TTL", "10"))
+# Hop-by-hop headers must not be forwarded (RFC 9110 §7.6.1) — forwarding
+# transfer-encoding or content-length of a rewritten body breaks clients.
+HOP_BY_HOP = {
+    "host", "content-length", "transfer-encoding", "connection",
+    "keep-alive", "proxy-authenticate", "proxy-authorization", "te",
+    "trailers", "upgrade",
+}
+
+UPSTREAMS = routing.UPSTREAMS
+# Replica health, refreshed by the watcher. Module state rather than aiohttp app
+# state: the application is frozen once the site is up.
+_UP: list[bool] = [True] * len(UPSTREAMS)
+
+
+async def health(session: ClientSession) -> web.Response:
+    results = await asyncio.gather(
+        *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
+        return_exceptions=True,
+    )
+    lines, up = [], 0
+    for url, res in zip(UPSTREAMS, results):
+        if isinstance(res, Exception) or res.status != 200:
+            lines.append(f"{url} DOWN")
+        else:
+            up += 1
+            lines.append(f"{url} UP")
+    return web.Response(status=200 if up else 503, text="\n".join(lines) + "\n")
+
+
+async def watch_health() -> None:
+    """Refresh the replica up/down table so routing never targets a dead one.
+
+    One HTTP request per replica per interval, in exchange for never handing a
+    client a 502 after a restart.
+    """
+    global _UP
+    session = ClientSession()
+    try:
+        while True:
+            res = await asyncio.gather(
+                *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
+                return_exceptions=True,
+            )
+            _UP = [not isinstance(r, Exception) and r.status == 200 for r in res]
+            await asyncio.sleep(HEALTH_TTL)
+    finally:
+        await session.close()
+
+
+async def proxy(request: web.Request) -> web.StreamResponse:
+    session: ClientSession = request.app["session"]
+    if request.path == "/health":
+        return await health(session)
+
+    body = await request.read()
+    headers = {k: v for k, v in request.headers.items() if k.lower() not in HOP_BY_HOP}
+
+    # Fail over before anything is written to the client. A replica that
+    # restarts is invisible to the health watcher for up to HEALTH_TTL, and a
+    # deploy restarts both replicas — without this retry a 10 s window of
+    # requests would 502 on a service that has a live card the whole time.
+    # vLLM's OpenAI endpoints are stateless per request, so re-sending a
+    # request that never reached an engine is safe; a request that failed
+    # mid-stream is not retried (that path is below).
+    tried: set[int] = set()
+    upstream = None
+    last_error = "no healthy frank-medium replica"
+    while len(tried) < len(UPSTREAMS):
+        idx = routing.route(body, request.remote, _UP)
+        if idx < 0 or idx in tried:
+            break
+        tried.add(idx)
+        target = UPSTREAMS[idx] + request.path_qs
+        try:
+            upstream = await session.request(
+                request.method, target, headers=headers, data=body or None,
+                timeout=ClientTimeout(total=None, sock_connect=10, sock_read=3600),
+            )
+            break
+        except Exception as exc:
+            # Demote at once: the watcher will confirm, but every request until
+            # then should skip this replica rather than pay the same timeout.
+            # Logged because a silent failover hides a card that is down.
+            _UP[idx] = False
+            last_error = f"{target} unreachable: {exc}"
+            print(f"proxy: failover — {target} unreachable ({exc}); retrying "
+                  f"another replica", flush=True)
+    if upstream is None:
+        # 503 when no replica was even reachable through the routing table,
+        # 502 when replicas were tried and all of them failed.
+        status = 502 if tried else 503
+        return web.Response(status=status, text=f"proxy: {last_error}\n")
+
+    out = web.StreamResponse(status=upstream.status, reason=upstream.reason)
+    for k, v in upstream.headers.items():
+        if k.lower() not in HOP_BY_HOP:
+            out.headers[k] = v
+    await out.prepare(request)
+    try:
+        async for chunk in upstream.content.iter_any():
+            await out.write(chunk)
+    except Exception:
+        # The client went away mid-stream (an aborted generation). Nothing to
+        # report: the engine sees its own cancellation.
+        pass
+    finally:
+        upstream.release()
+    with suppress(Exception):
+        await out.write_eof()
+    return out
+
+
+async def main() -> None:
+    app = web.Application(client_max_size=1024 ** 3)
+    app["session"] = ClientSession()
+    app.router.add_route("*", "/{tail:.*}", proxy)
+    runner = web.AppRunner(app)
+    await runner.setup()
+    await web.TCPSite(runner, "0.0.0.0", PORT).start()
+    print(f"proxy: :{PORT} -> {', '.join(UPSTREAMS)} routing={routing.ROUTING} "
+          f"key={routing.AFFINITY_BYTES}B known_max={routing.KNOWN_MAX}", flush=True)
+    watcher = asyncio.create_task(watch_health())
+    with suppress(asyncio.CancelledError):
+        await asyncio.gather(watcher)
+
+
+if __name__ == "__main__":
+    with suppress(KeyboardInterrupt):
+        asyncio.run(main())
@@ -0,0 +1,98 @@
+# Quadlet unit: frank-medium replica A (rootless, boot-persistent).
+#
+# Install path on dankserver: ~/.config/containers/systemd/frank-medium-a.container
+# Unit name: frank-medium-a.service (container name: frank-medium-a)
+#
+# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
+# tensor-parallel process across both R9700s (see frank-medium.container, kept
+# as the target). That shape cannot run on this host: RCCL's collective kernels
+# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
+# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
+# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
+# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
+# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
+# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
+# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
+# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
+# atomics. Result: every collective that touches GPU 2 dies at launch.
+# Two independent single-GPU replicas use both cards instead; the proxy unit
+# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
+#
+# SHAPE. vLLM binds 0.0.0.0:7771 on the host network directly; no PublishPort.
+# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
+# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
+# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
+# this shape instead of using the 2x7551-32624 pin, which is only valid at
+# TP=2/262144.
+#
+# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=0 (the first R9700). The
+# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
+# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
+# first and HIP indexes into the filtered list, so a single-valued pair
+# (ROCR=1,HIP=1) leaves an empty device set.
+#
+# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
+# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
+[Unit]
+Description=frank-medium replica A — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #1
+After=network-online.target
+
+[Container]
+ContainerName=frank-medium-a
+Image=localhost/radiance-vllm:0.9.3
+AddDevice=/dev/kfd
+AddDevice=/dev/dri
+GroupAdd=keep-groups
+# The upstream launcher's exact podman posture (rootless).
+PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
+Environment=HIP_VISIBLE_DEVICES=0
+Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
+Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
+Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
+Environment=SERVED_NAMES=frank-medium
+Environment=PORT=7771
+Environment=TP=1
+Environment=MAXSEQS=4
+Environment=CHUNK=8192
+Environment=MAXLEN=114688
+Environment=GPU_UTIL=0.98
+Environment=KV_MEM=0
+Environment=SPEC_METHOD=dflash
+Environment=SPEC=7
+# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
+Volume=/home/dank/models:/models
+# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
+Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
+# Compile cache. Shared with the other replica and with the TP=2 target: the
+# entries are content-addressed, so shapes simply add subdirectories.
+Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
+# The pinned radiance clone: patches applied at every container start.
+Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
+# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
+# shipped r4d.so predates the GDN overflow fix and NaNs this model.
+Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
+# Entrypoint + healthcheck from this checkout, mounted read-only.
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
+# The entrypoint must be the container command; the image's own entrypoint
+# (which the patched script execs at the end) must not run unpatched.
+PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
+# 200 only once the engine serves; during first-start compilation the
+# container is legitimately unhealthy. HealthCmd is split on spaces into an
+# argv array by the podman 5.x v1 generator, hence the file-based probe.
+HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
+HealthInterval=60s
+HealthTimeout=30s
+HealthRetries=3
+HealthStartPeriod=1800s
+Pull=never
+
+[Service]
+# First start compiles Triton/inductor kernels (several minutes) before the
+# engine loads ~20 GiB of weights.
+TimeoutStartSec=1800
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=default.target
@@ -0,0 +1,98 @@
+# Quadlet unit: frank-medium replica B (rootless, boot-persistent).
+#
+# Install path on dankserver: ~/.config/containers/systemd/frank-medium-b.container
+# Unit name: frank-medium-b.service (container name: frank-medium-a)
+#
+# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
+# tensor-parallel process across both R9700s (see frank-medium.container, kept
+# as the target). That shape cannot run on this host: RCCL's collective kernels
+# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
+# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
+# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
+# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
+# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
+# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
+# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
+# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
+# atomics. Result: every collective that touches GPU 2 dies at launch.
+# Two independent single-GPU replicas use both cards instead; the proxy unit
+# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
+#
+# SHAPE. vLLM binds 0.0.0.0:7772 on the host network directly; no PublishPort.
+# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
+# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
+# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
+# this shape instead of using the 2x7551-32624 pin, which is only valid at
+# TP=2/262144.
+#
+# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=1 (the second R9700). The
+# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
+# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
+# first and HIP indexes into the filtered list, so a single-valued pair
+# (ROCR=1,HIP=1) leaves an empty device set.
+#
+# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
+# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
+[Unit]
+Description=frank-medium replica B — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #2
+After=network-online.target
+
+[Container]
+ContainerName=frank-medium-b
+Image=localhost/radiance-vllm:0.9.3
+AddDevice=/dev/kfd
+AddDevice=/dev/dri
+GroupAdd=keep-groups
+# The upstream launcher's exact podman posture (rootless).
+PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
+Environment=HIP_VISIBLE_DEVICES=1
+Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
+Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
+Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
+Environment=SERVED_NAMES=frank-medium
+Environment=PORT=7772
+Environment=TP=1
+Environment=MAXSEQS=4
+Environment=CHUNK=8192
+Environment=MAXLEN=114688
+Environment=GPU_UTIL=0.98
+Environment=KV_MEM=0
+Environment=SPEC_METHOD=dflash
+Environment=SPEC=7
+# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
+Volume=/home/dank/models:/models
+# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
+Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
+# Compile cache. Shared with the other replica and with the TP=2 target: the
+# entries are content-addressed, so shapes simply add subdirectories.
+Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
+# The pinned radiance clone: patches applied at every container start.
+Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
+# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
+# shipped r4d.so predates the GDN overflow fix and NaNs this model.
+Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
+# Entrypoint + healthcheck from this checkout, mounted read-only.
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
+# The entrypoint must be the container command; the image's own entrypoint
+# (which the patched script execs at the end) must not run unpatched.
+PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
+# 200 only once the engine serves; during first-start compilation the
+# container is legitimately unhealthy. HealthCmd is split on spaces into an
+# argv array by the podman 5.x v1 generator, hence the file-based probe.
+HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
+HealthInterval=60s
+HealthTimeout=30s
+HealthRetries=3
+HealthStartPeriod=1800s
+Pull=never
+
+[Service]
+# First start compiles Triton/inductor kernels (several minutes) before the
+# engine loads ~20 GiB of weights.
+TimeoutStartSec=1800
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=default.target
@@ -0,0 +1,61 @@
+# Quadlet unit: frank-medium proxy (rootless, boot-persistent).
+#
+# Install path on dankserver: ~/.config/containers/systemd/frank-medium-proxy.container
+# Unit name: frank-medium-proxy.service (container name: frank-medium-proxy)
+#
+# WHAT IT IS. A prefix-affinity reverse proxy in front of the two single-GPU
+# frank-medium replicas (:7771 on R9700 #1, :7772 on R9700 #2). It preserves
+# the fleet contract: one endpoint on host port 7770 answering as the served
+# model name frank-medium, OpenAI /v1, streaming included.
+#
+# WHY IT EXISTS. TP=2 — one vLLM process spanning both cards — is impossible
+# on this host (see frank-medium-a.container and the runbook: RCCL's kernels
+# need PCIe atomics, and the second card's chipset path advertises
+# AtomicOpsCap: Routing-). Two replicas are how both cards get used.
+#
+# ROUTING. Requests are keyed on a hash of their leading 4 KiB of prompt text,
+# so a conversation family — an agent loop re-sending the same system prompt,
+# tool list and file context every turn — keeps landing on the same replica,
+# where --enable-prefix-caching has that prefix warm. Plain round-robin threw
+# half of those cache hits away. A replica that is down is skipped, and
+# ROUTING=roundrobin restores the old behaviour for A/B measurement. /health
+# answers 200 while at least one replica is up and names the ones that are
+# down. The proxy runs on host networking so it can bind 7770 and reach the
+# replicas on loopback.
+#
+# The script is a few dozen lines of aiohttp, mounted read-only out of
+# /home/dank/serverAppConfigs (deploy.sh pulls that checkout before dispatch).
+# It reuses the same image as the replicas: aiohttp is already there and no
+# extra image has to be pulled on this host.
+[Unit]
+Description=frank-medium proxy — prefix-affinity in front of the two single-GPU replicas
+After=network-online.target
+Wants=frank-medium-a.service frank-medium-b.service
+After=frank-medium-a.service frank-medium-b.service
+
+[Container]
+ContainerName=frank-medium-proxy
+Image=localhost/radiance-vllm:0.9.3
+PodmanArgs=--network=host
+Environment=PORT=7770
+Environment=UPSTREAMS=http://127.0.0.1:7771,http://127.0.0.1:7772
+# The proxy is two files: routing.py holds the replica-selection policy, the
+# server holds the HTTP plumbing. Both are mounted because the server imports
+# the policy by name from its own directory (/usr/local/bin is sys.path[0]).
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/routing.py:/usr/local/bin/routing.py:ro
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/affinity-proxy.py:/usr/local/bin/affinity-proxy.py:ro
+Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
+PodmanArgs=--entrypoint /usr/local/bin/affinity-proxy.py
+HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
+HealthInterval=60s
+HealthTimeout=30s
+HealthRetries=3
+HealthStartPeriod=60s
+Pull=never
+
+[Service]
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=default.target
@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+"""Routing policy for the frank-medium proxy: which replica serves a request.
+
+Kept apart from the HTTP plumbing in affinity-proxy.py so the decisions can be
+tested without aiohttp, a network or a GPU — the first version of these rules
+shipped with a bug that only appeared on a continuation request, and a test
+that loads this module catches that class of mistake directly.
+
+Two cards, two behaviours, and the workload decides which is right:
+
+* A **single-turn** request is one member of a batch (a benchmark run, a burst
+  of parallel subagents). Spreading those over both replicas lets both cards
+  prefill at once; measured on this host, a 16-request burst of 15k-token
+  prompts finishes in 18.6 s spread versus 30.1 s pinned.
+* A **continuation** re-sends a prefix the replica has already prefilled. Pinned
+  to that replica the turn costs a cache lookup; spread, half the turns pay a
+  cold prefill again (measured: 1096 ms versus 2676 ms mean warm-turn TTFT on
+  90k-token contexts).
+
+ROUTING picks the strategy; the default `adaptive` applies both rules above.
+`affinity` and `roundrobin` exist so the choice can be re-measured, and are not
+production settings.
+"""
+
+from __future__ import annotations
+
+import collections
+import hashlib
+import itertools
+import json
+import os
+
+UPSTREAMS = [u.strip() for u in os.environ.get(
+    "UPSTREAMS", "http://127.0.0.1:7771,http://127.0.0.1:7772").split(",") if u.strip()]
+ROUTING = os.environ.get("ROUTING", "adaptive").strip().lower()
+# Leading bytes of prompt text that identify a conversation's shared prefix.
+# 4 KiB covers a system prompt plus the start of the first user turn, which is
+# where agent loops keep the text they resend every turn.
+AFFINITY_BYTES = int(os.environ.get("AFFINITY_BYTES", "4096"))
+# Prefixes remembered for pinning. Bounded so a long-lived proxy cannot grow
+# without limit; 4096 conversations is far beyond what this fleet runs at once.
+KNOWN_MAX = int(os.environ.get("KNOWN_MAX", "4096"))
+
+_counter = itertools.count()
+# digest -> replica index, insertion-ordered for O(1) LRU eviction. A prefix is
+# remembered as soon as any request carries it; only a continuation is pinned
+# there, because only a continuation can profit from the cache entry.
+_known: "collections.OrderedDict[bytes, int]" = collections.OrderedDict()
+
+
+def affinity_key(body: bytes, remote: str | None) -> tuple[bytes, bool]:
+    """Key a request by its shared prefix, and report whether it continues one.
+
+    Chat and completions bodies both carry their prompt text in JSON, so the
+    leading bytes of that text are the prefix; a body we cannot parse (or an
+    endpoint with no prompt, e.g. /v1/models) falls back to the client address,
+    which still beats nothing for a single-client loop.
+    """
+    fallback = (remote or "anonymous").encode()
+    try:
+        payload = json.loads(body)
+    except Exception:
+        return fallback, False
+
+    text = ""
+    continuation = False
+    if isinstance(payload, dict):
+        if isinstance(payload.get("prompt"), str):
+            text = payload["prompt"]
+        elif isinstance(payload.get("messages"), list):
+            continuation = len(payload["messages"]) > 1
+            parts = []
+            for msg in payload["messages"]:
+                if isinstance(msg, dict):
+                    content = msg.get("content")
+                    if isinstance(content, str):
+                        parts.append(content)
+                    elif isinstance(content, list):  # multimodal content blocks
+                        parts.extend(
+                            blk.get("text", "") for blk in content
+                            if isinstance(blk, dict) and isinstance(blk.get("text"), str)
+                        )
+            text = "\n".join(parts)
+    if not text:
+        return fallback, continuation
+    return text[:AFFINITY_BYTES].encode("utf-8", "replace"), continuation
+
+
+def pick(key: bytes, continuation: bool, up: list[bool]) -> int:
+    """Replica for a request, or -1 when none is available.
+
+    Down replicas are never selected — a rollout must not send traffic to an
+    engine that is still compiling its graphs — and a pin whose replica is down
+    is re-established on a live one rather than kept pointing at a corpse.
+    """
+    digest = hashlib.sha1(key).digest()
+
+    if ROUTING == "roundrobin":
+        start = next(_counter) % len(UPSTREAMS)
+    elif ROUTING == "adaptive":
+        entry = _known.get(digest)
+        if continuation and entry is not None and up[entry]:
+            _known.move_to_end(digest)
+            return entry
+        # Anything that is not a continuation spreads by counter — including a
+        # batch of requests that happen to share one prefix. Hashing the prefix
+        # would send every member of such a batch to whichever card the hash
+        # names (measured: 29.8 s for a 16-request burst, against 18.6 s when
+        # spread). The counter is only advanced by these requests, so batch
+        # members alternate across the cards.
+        start = next(_counter) % len(UPSTREAMS)
+    else:  # affinity
+        start = int.from_bytes(digest[:8], "big") % len(UPSTREAMS)
+
+    for offset in range(len(UPSTREAMS)):
+        idx = (start + offset) % len(UPSTREAMS)
+        if up[idx]:
+            if ROUTING == "adaptive":
+                _known[digest] = idx
+                _known.move_to_end(digest)
+                if len(_known) > KNOWN_MAX:
+                    _known.popitem(last=False)
+            return idx
+    return -1
+
+
+def route(body: bytes, remote: str | None, up: list[bool]) -> int:
+    """Convenience wrapper: decode the request body and choose a replica."""
+    return pick(*affinity_key(body, remote), up)
@@ -43,7 +43,7 @@ After=network.target

 [Container]
 ContainerName=calibre-server
-Image=lscr.io/linuxserver/calibre:9.17.0@sha256:b198fc03a6abc6be7e251a9958e50e74634b00e447c7b25e92c191c8077ba378
+Image=lscr.io/linuxserver/calibre:9.15.0
 Network=pasta
 EnvironmentFile=/home/dank/Containers/calibre-server/env
 Environment=PUID=1000
@@ -16,7 +16,7 @@ After=network.target

 [Container]
 ContainerName=calibre-web
-Image=lscr.io/linuxserver/calibre-web:0.6.27-ls407@sha256:470b4ed9a7d6ea1475fef17dbcf957cb60880910296d226bd0dca652c05f68ca
+Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402
 Network=pasta
 Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
 Environment=LANG=en_US.UTF-8
@@ -94,12 +94,34 @@ services:
      timeout: 10s
      retries: 3

-  # Jellyfin left this file when it became a Quadlet: the container is owned by
-  # dankserver/jellyfin/quadlet/jellyfin.container, its data roots are under
-  # /home/dank/Containers/jellyfin, and its library arrives through the SSD
-  # overlay at /var/lib/media-meta/view (scripts/media-view.sh). The block that
-  # was here named /opt/jellyfin/* and a read-only array bind — a second owner for
-  # the container name `jellyfin`, wrong on every path. One owner per container.
+  # ═══════════════════════════════════════════════
+  # Jellyfin — media server (movies, TV, music)
+  # ═══════════════════════════════════════════════
+  jellyfin:
+    image: jellyfin/jellyfin:latest
+    container_name: jellyfin
+    restart: unless-stopped
+    ports:
+      - "8096:8096/tcp"
+      - "8920:8920/tcp"
+      - "7359:7359/udp"
+      - "1900:1900/udp"
+    environment:
+      - JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
+      - DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
+    volumes:
+      - /opt/jellyfin/config:/config
+      - /opt/jellyfin/cache:/cache
+      - /mnt/entropy:/data:ro
+    devices:
+      - /dev/dri:/dev/dri
+    group_add:
+      - "992"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3

  # ═══════════════════════════════════════════════
  # Calibre-Web — e-book web frontend
@@ -21,13 +21,7 @@ After=network.target

 [Container]
 ContainerName=copyparty
-# Digest-pinned: upstream stopped publishing version tags after 1.19.15, so
-# the pin rides :latest@digest (the tag in the reference is cosmetic; the
-# digest is the pull). Pinned 2026-10-21 at the build whose label is
-# 1.20.26 - closes GHSA-3c88-p299-7hvr (1.20.25) and GHSA-hg99-jqfp-rf76
-# (1.20.26); the previously running build was the rolling :latest at
-# 1.20.24 (issue #73 on command-center).
-Image=ghcr.io/9001/copyparty-ac:latest@sha256:7f77459160a4bf34fdca79f2965dff4f0c1d7069a3d9cdfb3632acb041dd2f6a
+Image=ghcr.io/9001/copyparty-ac:latest
 Network=dankserver_appnet
 EnvironmentFile=/home/dank/Containers/copyparty/env
 Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD} -v /w::r:ra,dank
@@ -36,9 +30,6 @@ Volume=/home/dank/Containers/copyparty/cfg:/cfg
 PublishPort=3923:3923

 [Service]
-# /w is the array: serving the pre-mount directory instead of the volume shows an
-# empty share and nothing complains, so the mount has to answer first.
-ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /mnt/entropy
 EnvironmentFile=/home/dank/Containers/copyparty/env
 Restart=always
 RestartSec=5
@@ -1 +0,0 @@
-entropy	UUID=cd86c728-459e-4f3a-bcc2-41ecf84685ec	/etc/entropy	nofail,luks
@@ -1,18 +0,0 @@
-# /etc/fstab: static file system information.
-#
-# Use 'blkid' to print the universally unique identifier for a
-# device; this may be used with UUID= as a more robust way to name devices
-# that works even if disks are added and removed. See fstab(5).
-#
-# <file system> <mount point>   <type>  <options>       <dump>  <pass>
-# / was on /dev/nvme0n1p2 during curtin installation
-/dev/disk/by-uuid/4f3731d2-fb7b-48e7-a79f-0e33cb8a4162 / ext4 defaults 0 1
-# /boot/efi was on /dev/nvme0n1p1 during curtin installation
-/dev/disk/by-uuid/183C-442D /boot/efi vfat defaults 0 1
-#/dev/disk/by-label/archive-one /mnt/archive-one auto nosuid,nodev,nofail,x-gvfs-show 0 0
-#/dev/disk/by-label/archive-two /mnt/archive-two auto nosuid,nodev,nofail,x-gvfs-show 0 0
-# /dev/md0 is a LUKS container, not a filesystem; its ext4 is mounted via
-# /dev/mapper/entropy on the next line. Mounting /dev/md0 directly fails.
-#/dev/md0 /mnt/md0 ext4 auto,nofail,nosuid,nodev 0 0 
-/dev/mapper/entropy /mnt/entropy ext4 auto,nosuid,nofail,nodev,x-gvfs-show 0 0
-/swapfile none swap sw 0 0
@@ -1,30 +0,0 @@
-# /etc/systemd/system/var-lib-media\x2dmeta-view.mount
-# (repo copy: dankserver/host/systemd/media-meta-view.mount — systemd names a
-# mount unit after its ESCAPED path, so the host file carries the \x2d spelling)
-#
-# Installed by `scripts/media-view.sh install` — the repo copy is the source of
-# truth (fleet law: never edit host state by hand).
-#
-# An OverlayFS merge that shows the media library as one tree while splitting
-# where its bytes live: lowerdir is the RAID6 array (the source of truth, never
-# written), upperdir is a directory on the NVMe that receives every write. The
-# two media servers bind the result at /var/lib/media-meta/view, so Jellyfin's
-# sidecars and artwork are SSD bytes — which is what makes them fast to browse
-# and what keeps Plex's NFO agent off the spinners — while the media itself stays
-# on the array untouched.
-#
-# RequiresMountsFor is load-bearing: without the array mounted, lowerdir is an
-# empty directory and the view would present an empty library as truth.
-
-[Unit]
-Description=SSD-backed overlay view of the media library (array + NVMe metadata layer)
-RequiresMountsFor=/mnt/entropy
-
-[Mount]
-What=overlay
-Where=/var/lib/media-meta/view
-Type=overlay
-Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work
-
-[Install]
-WantedBy=local-fs.target
@@ -1,14 +0,0 @@
-# The io controller is what makes IOWeight= mean anything, and systemd does not
-# enable it below `user.slice` by default: with io accounting off, `io` never
-# appears in `user.slice/cgroup.subtree_control`, so no rootless container can
-# have an `io.weight` file and `IOWeight=` in a quadlet is silently inert.
-# Measured on Dankserver 2026-10-31: setting this property flipped
-# `user.slice/cgroup.subtree_control` from `cpu memory pids` to `cpu io memory
-# pids` immediately, with nothing restarted.
-#
-# This is half the fix; the other half is delegating io to user@1000.service
-# (user@1000.service.d/io-delegate.conf). Both must be in place before a reboot,
-# because the delegation is read when the user manager starts. Law and the
-# verification command: [[Quadlet Lifecycle]].
-[Slice]
-IOAccounting=yes
@@ -1,12 +0,0 @@
-# The system manager gives the user manager cpu/memory/pids by default (systemd
-# 259: `systemctl show user@1000.service -p DelegateControllers` → `cpu memory
-# pids`). Without `io` in that set no `io.weight` file can exist anywhere below
-# `user.slice`, so `IOWeight=` in a rootless quadlet is silently inert — the same
-# class of trap as `RequiresMountsFor=`. Granting the controller here is what
-# makes the media servers' I/O share real.
-#
-# Inert until `user@1000.service` starts again, i.e. the next reboot (restarting
-# it would kill every rootless container on the host). Verify after the reboot
-# with the check in [[Quadlet Lifecycle]].
-[Service]
-DelegateControllers=cpu memory pids io
@@ -4,9 +4,8 @@
 # Unit name: jellyfin.service (podman 5.x drops the container- prefix)
 #
 # Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
-# cutover. The library is seen through /var/lib/media-meta/view, the SSD-backed
-# overlay view (scripts/media-view.sh): the array stays read-only and untouched,
-# while the sidecars this server writes land on the NVMe. Every
+# cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
+# anywhere writes to the entropy volume. Every
 # creation-time environment variable of the compose-era container is kept as
 # an Environment= line (zero drift); none of them are secrets, so this unit
 # needs no EnvironmentFile.
@@ -17,7 +16,7 @@ After=network.target

 [Container]
 ContainerName=jellyfin
-Image=ghcr.io/jellyfin/jellyfin:latest@sha256:fea863d0cd9ec445985d9c75fc6ed64e38a1d2766cf0c250a33d553f4fc1b771
+Image=ghcr.io/jellyfin/jellyfin:latest
 Network=dankserver_appnet
 Environment=DEBIAN_FRONTEND=noninteractive
 Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
@@ -42,11 +41,7 @@ PublishPort=7359:7359/udp
 PublishPort=1900:1900/udp
 Volume=/home/dank/Containers/jellyfin/config:/config
 Volume=/home/dank/Containers/jellyfin/cache:/cache
-# The view, writable: Jellyfin is the metadata writer (movie.nfo / tvshow.nfo /
-# episode.nfo + artwork) and those bytes belong on the SSD. The gate in [Service]
-# is the guard: started against a plain, unmounted view directory a scan decides
-# the library is empty and deletes what it cannot see.
-Volume=/var/lib/media-meta/view:/data
+Volume=/mnt/entropy:/data:ro
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 HealthCmd=curl -f http://localhost:8096/health
@@ -55,19 +50,8 @@ HealthTimeout=10s
 HealthRetries=3

 [Service]
-# Refuse to start until the bind source is a real mount — see the header of
-# scripts/wait-for-mount.sh for the blind-boot incident this prevents.
-ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /var/lib/media-meta/view /mnt/entropy
 Restart=always
 RestartSec=5

-# A library scan reads the array for minutes on end. Weighted LOW (default is
-# 100) so that when the array is contended a scan loses to playback and to every
-# other container this user runs. Proportional share, not a cap: it changes
-# nothing while the disk is idle. 10 is a convention — the requirement is only
-# that it sit far below 100 — and it is inert until user@1000.service is
-# delegated the io controller ([[Quadlet Lifecycle]]).
-IOWeight=10
-
 [Install]
 WantedBy=default.target
@@ -1,39 +0,0 @@
-# Quadlet unit: cadvisor (rootless, boot-persistent) — DANKSERVER.
-#
-# Install path on dankserver: ~/.config/containers/systemd/cadvisor.container
-# Unit name: cadvisor.service (container name: cadvisor)
-#
-# The Dankserver twin of dankclaw/observability/quadlet/cadvisor.container.
-# Same rootless shape, same --cgroupns=host requirement (see that file for the
-# full story), only the LAN address differs: this host publishes on
-# 192.168.0.235:8080 so Prometheus on Dankclaw can scrape it over the real
-# network (cross-host, so the host.containers.internal trick does not apply).
-# It reports the media/library stack (romm, plex, jellyfin, copyparty, calibre,
-# tautulli) by their quadlet unit names via the cgroup path.
-
-[Unit]
-Description=cAdvisor container metrics (Dankserver)
-After=network-online.target
-Wants=network-online.target
-
-[Container]
-ContainerName=cadvisor
-Image=gcr.io/cadvisor/cadvisor:v0.49.1@sha256:3cde6faf0791ebf7b41d6f8ae7145466fed712ea6f252c935294d2608b1af388
-PodmanArgs=--pid=host --cgroupns=host
-PublishPort=127.0.0.1:8080:8080
-PublishPort=192.168.0.235:8080:8080
-Volume=/sys:/sys:ro
-Volume=/:/rootfs:ro
-Exec=--housekeeping_interval=30s
-HealthCmd=wget --spider -q http://localhost:8080/healthz
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=40s
-
-[Service]
-Restart=always
-RestartSec=10
-
-[Install]
-WantedBy=default.target
@@ -37,7 +37,7 @@ Wants=network-online.target

 [Container]
 ContainerName=node-exporter
-Image=docker.io/prom/node-exporter:latest@sha256:1b4e4438faca4dd7e001dd445d161a4a2091b0fededa84093b3a8dfeae1f1be0
+Image=docker.io/prom/node-exporter:latest
 Network=host
 PodmanArgs=--pid=host
 Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.235:9100
@@ -4,33 +4,19 @@ After=network-online.target
 Wants=network-online.target

 [Container]
-Image=lscr.io/linuxserver/plex:1.43.5.11079-b3b13e506-ls330@sha256:01a53d9c7adcbb3701fbb0a4230c0a37c85453759b6b05a67356c3c7a09de08d
+Image=lscr.io/linuxserver/plex:1.43.4.10903-e5521bd8c-ls325
 ContainerName=plex
 Network=dankserver_appnet:alias=plex
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
 Volume=/var/lib/plexmediaserver:/config
-# The container path stays /mnt/entropy on purpose: every path Plex recorded in
-# its own database is written against it, so swapping the HOST source of this bind
-# to the overlay view re-points Plex at the sidecars without touching its DB.
-# Read-only: Plex consumes the NFO files Jellyfin writes and writes no metadata.
-Volume=/var/lib/media-meta/view:/mnt/entropy:ro
+Volume=/mnt/entropy:/mnt/entropy:ro
 PublishPort=32400:32400

 [Service]
-# Refuse to start until the bind source is a real mount: Plex blind is how paths
-# get marked missing in its own database. Both the view and the array under it
-# must answer — scripts/wait-for-mount.sh names the 2026-10-31 incident.
-ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /var/lib/media-meta/view /mnt/entropy
 Restart=always
 RestartSec=5

-# See the jellyfin unit for the reasoning: a section scan gets a LOW weight
-# (default 100) so a walk of 50k files loses the array to playback and to every
-# other container under this user. Inert until user@1000.service holds the io
-# controller ([[Quadlet Lifecycle]]).
-IOWeight=10
-
 [Install]
 WantedBy=default.target
@@ -14,7 +14,7 @@ After=network.target

 [Container]
 ContainerName=romm-mariadb
-Image=docker.io/library/mariadb:11@sha256:c4124d1afa875e47f2892edf26b482fa54f796e2f342d3717a04ca091a22a9bd
+Image=docker.io/library/mariadb:11
 Network=dankserver_appnet:alias=mariadb
 EnvironmentFile=/home/dank/Containers/romm/env
 Environment=MARIADB_DATABASE=romm
@@ -12,7 +12,7 @@ After=network.target

 [Container]
 ContainerName=romm-redis
-Image=docker.io/library/redis:7-alpine@sha256:8da19c7399dca900c6822a1cf30f9d2285585e2c4b457ff4095b0f4500054d3f
+Image=docker.io/library/redis:7-alpine
 Network=dankserver_appnet:alias=redis
 Volume=/home/dank/Containers/romm/redis:/data
 HealthCmd=redis-cli ping
@@ -18,7 +18,7 @@ After=romm-mariadb.service romm-redis.service network.target

 [Container]
 ContainerName=romm
-Image=ghcr.io/rommapp/romm:latest@sha256:8677fb7b81b1d3db20b849e989a1dadc2a11159e51eb34a2acdcb01102274edf
+Image=ghcr.io/rommapp/romm:latest
 Network=dankserver_appnet
 EnvironmentFile=/home/dank/Containers/romm/env
 Environment=DB_HOST=mariadb
@@ -4,7 +4,7 @@ After=network-online.target
 Wants=network-online.target

 [Container]
-Image=lscr.io/linuxserver/tautulli:v2.18.2-ls250@sha256:529c7eabcc40554c1dc48f33f1f3cc3b1438462d73e0ec2dac8ac0b66aeeb8d4
+Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245
 ContainerName=tautulli
 Network=dankserver_appnet:alias=tautulli
 Environment=PUID=0
@@ -19,7 +19,7 @@ Description=Gluetun VPN tunnel (FrankBook-Pro podman machine)

 [Container]
 ContainerName=gluetun
-Image=docker.io/qmcgaw/gluetun:latest@sha256:c2ee4191762fca6b6a3c3359a731198a95d9c9af4db32d466e223bf3a84e779b
+Image=docker.io/qmcgaw/gluetun:latest
 AddCapability=NET_ADMIN
 AddDevice=/dev/net/tun:/dev/net/tun
 Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
@@ -15,7 +15,7 @@ After=gluetun.service

 [Container]
 ContainerName=lidarr
-Image=lscr.io/linuxserver/lidarr:latest@sha256:d8549ff9ea29b4015c32257360652c1000c31ea471a2b3f0835ae77292faf35b
+Image=lscr.io/linuxserver/lidarr:latest
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -24,7 +24,7 @@ After=gluetun.service

 [Container]
 ContainerName=prowlarr
-Image=lscr.io/linuxserver/prowlarr:latest@sha256:b3b00047ed20f6ce2b16780145cb6a0acb51c9e28ea6aa6f5dcd33c7034f1797
+Image=lscr.io/linuxserver/prowlarr:latest
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -29,7 +29,7 @@ Network=gluetun.container
 # Pinned to the last 4.x line: Readarr's retired qB client requires the
 # qB 4.x login response (HTTP 200, body "Ok."); qB 5.x answers 204 with an
 # empty body, which fails Readarr's auth check. All other clients accept 4.x.
-Image=lscr.io/linuxserver/qbittorrent:4.6.7@sha256:55f15d44396315551f87294a176efae733b16e283f38980308e46073950257c6
+Image=lscr.io/linuxserver/qbittorrent:4.6.7
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -15,7 +15,7 @@ After=gluetun.service

 [Container]
 ContainerName=radarr
-Image=lscr.io/linuxserver/radarr:latest@sha256:fc0ae3c50f8128b97e84381903ec8d6845c3a569641cb7117c124376cccb57c1
+Image=lscr.io/linuxserver/radarr:latest
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -19,7 +19,7 @@ After=gluetun.service

 [Container]
 ContainerName=readarr
-Image=lscr.io/linuxserver/readarr:amd64-1.0.980@sha256:a95e82dbb0eb2bb43de4c150f1dab9dd821306c67ea0b418983a957e731f2b61
+Image=lscr.io/linuxserver/readarr:amd64-1.0.980
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -16,7 +16,7 @@ After=gluetun.service

 [Container]
 ContainerName=sonarr
-Image=lscr.io/linuxserver/sonarr:latest@sha256:889dcc58c6aea859033985894b519b2e6b1e55d92527ed123f9f52e42ae7c97b
+Image=lscr.io/linuxserver/sonarr:latest
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
@@ -1,319 +0,0 @@
-# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
-
-Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
-Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
-2026-10, and whether any of them can serve a speech-synthesis model.
-
-> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
-> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
-> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
-> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
-
-Convention: vendor claims carry URL + the version/date they hold for.
-**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
-vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
-
-## 0. Answers to the three questions Main ranked
-
-**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
-host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
-lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
-*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
-release note "New TTS model support: neutts-air, neutts-nano" —
-<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
-published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
-2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
-Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
-**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
-`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
-`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
-reference-audio encoder**. [INFERENCE from HF file lists]
-
-**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
-**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
-
-**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
-ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
-generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
-lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
-
-## 1. Landscape: every way to reach the XDNA2 array on Linux
-
-| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
-|---|------|---------|------------------------------|------------------|
-| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
-| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
-| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
-| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
-| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
-| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
-| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
-| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
-
-Sources: §2–§6 below.
-
-## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
-
-* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
-  v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
-  Kraken, and Gorgon Point)."
-  <https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
-* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
-  generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
-  `amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
-  `amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
-  run` goes through **XRT** and fails with `No such device with index '0'` unless
-  `xrt-smi examine` lists the NPU; `memlock` must be raised.
-  <https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
-  2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
-  distro package on the host **[host]**.
-* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
-  auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
-  `v1/audio/transcriptions`, `v1/embeddings`. **There is no
-  `/v1/audio/speech`.**
-  <https://fastflowlm.com/docs/instructions/server/openapi/>
-* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
-  Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
-  **Whisper — card states "Type: Speech-to-Text (ASR)"**
-  (<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
-  Hy-MT2, MiniCPM. No speech-synthesis entry.
-* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
-  ("Internet access to HuggingFace is required to download the optimized model
-  kernels"). The only in-tree engine build is Gemma-4E
-  (`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
-  → **A new TTS architecture cannot be added to `flm` by us.**
-
-## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
-
-* **Install channel (important correction to the pypi.amd.com evidence).** The pip
-  index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
-  win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
-  single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
-  as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
-  account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
-  → HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
-  (**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
-  EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
-  serves it directly. Verified by me with curl 2026-10-19. I did not finish
-  downloading 7.8 GiB, so the tarball's *contents* list is unverified —
-  `[INFERENCE]` only where noted.)
-* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
-  `https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
-  contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
-  `xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
-  download + unpack). **The plugin deb ships firmware for
-  `/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
-  NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
-  `libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
-  source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
-  driver + XRT shim + firmware only.
-* **Our device is officially STX.** AMD's APU table
-  (<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
-  `0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
-  = PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
-  column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
-* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
-  2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
-  "amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
-  has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
-  reports the device as **NPU Strix Halo**.
-* Linux install recipe (1.8.0 docs
-  <https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
-  **Python 3.12** (`apt install python3.12 python3.12-venv`),
-  `libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
-  `LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
-  `quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
-  Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
-  flows. "Linux currently supports NPU only flow" (no iGPU EP).
-  Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
-  must be built with a 3.12 interpreter and the packages are 24.04-built
-  `[INFERENCE]`.
-* LLM flow on Linux
-  (<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
-  ready-to-run models from HF, `deployment/lib`,
-  `RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
-  run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
-  **"Model generation is not supported in current release"** (repeated in
-  relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
-  Linux"); needs `memlock unlimited`.
-  Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
-  (Phi-3.5-mini, 4K model).
-* Third-party confirmation of the OS split: Lemonade's backend matrix
-  (<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
-  fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
-  `ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
-  text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
-  **no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
-  Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
-
-## 4. Can we compile our own graph on Linux? (the no-recipe path)
-
-Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
-ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
-caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
-"precompile the EP context model using Python and deploy it using a C++ program"
-is explicitly supported); EP auto-partitions unsupported nodes to CPU;
-BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
-BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
-"general asymmetric quantization … to enable third-party quantized models";
-**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
-(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
-cache dirs are not portable across EP or NPU-driver versions.
-
-Operator coverage is the hard gate
-(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
-Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
-
-* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
-  (absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
-  `CumSum` (BF16/XINT8), `STFT` (XINT8 only).
-* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
-  table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
-  XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
-  the **LLM/OGA flow** (whose op set is exactly
-  `SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
-  (W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
-  `Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
-* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
-  `NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
-  `RMSNormalization` → graphs that emit them must be rewritten before they
-  partition onto the NPU. [INFERENCE from absence in the table.]
-* Quantization is a separate tool, not in the driver package: **AMD Quark**
-  (`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
-  softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
-  on pypi.org under the obvious names **[host probe]** and ships from AMD's
-  gated/pip-private channel.
-* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
-  OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
-  speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
-  Windows box) in the loop.
-* Practical shape cost: the neutts artifacts carry **one compiled variant per
-  padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
-  prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
-  artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
-
-## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
-
-NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
-clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
-codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
-`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
-require **pre-encoded references** — these ship with the repo for all bundled
-voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
-active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
-(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
-codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
-(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
-2026-08-26, `qwen2`, GGUF + safetensors).
-
-Consequences for the fleet spike:
-
-1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
-   work either way**. Our `reference.wav` voices survive only if we run the
-   NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
-   once per voice and then feed pre-encoded codes. That is a one-time offline
-   step per voice; after that the compiled graph takes codes/prompt prefix as
-   ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
-   speaker prefix. `[INFERENCE]`
-2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
-   Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
-   roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
-   `[INFERENCE — different model; must be measured.]`
-3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
-   NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
-   3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
-4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
-   only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
-   (<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
-   tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
-   `Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
-   — no TTS directory. Serving TTS therefore means writing our own OGA host
-   (or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
-
-## 6. Two graphs on one NPU, and the iGPU hazard
-
-* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
-  (in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
-  npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
-  context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
-  (`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
-  `fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
-  So halogen's armed models and a TTS graph can hold separate contexts
-  simultaneously; they contend for columns and OPC share, they are not mutually
-  exclusive.
-* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
-  contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
-  resource distribution across applications"; 1.3 "Idle detection and suspension
-  of contexts", "Rebalance feature for AIE hardware resource optimization";
-  1.0 known issue "Support for multiple applications is limited to up to eight"
-  (all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
-* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
-  XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
-  1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
-  — the vendor README warns that a stale/mismatched `npu.sbin` causes
-  `ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
-  than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
-  `EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
-  fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
-  under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
-  the whole host.
-* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
-  vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
-  optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
-  system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
-  knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
-  (<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
-  <https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
-  The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
-  fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
-
-## 7. Paths I checked and ruled out (so nobody re-walks them)
-
-* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
-  `ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
-  generation. Not usable on Dankclaw.
-* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
-  AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
-  (<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
-  EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
-  deployment dir, per llm_linux).
-* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
-  (`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
-  NPU; migration means a different runtime, not a flag.
-* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
-  by install cost, and it is not a door for TTS.
-* **IRON / mlir-aie** is the only open toolchain that documents running on the
-  Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
-  AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
-  Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
-  `ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
-  required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
-  fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
-  optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
-  hand-written per-op tile programming, not a model compiler: writing a TTS model
-  here is a multi-month engineering project, not a migration.
-* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
-
-## 8. Recommendation implied by the above (for Main's spike)
-
-Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
-+ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
-distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
-"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
-each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
-is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
-the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
-serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
-
-## 9. Loose ends (not verified; I stopped on Main's stand-down)
-
-* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
-  the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
-  and which Python wheels it carries. My statement that non-LLM compilation works
-  on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
-  … NPU"), not on the package contents. Download was killed mid-flight.
-* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
-  runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
-  documented anywhere in 1.8.0 docs).
-* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
-  from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).
@@ -1,438 +0,0 @@
-# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability
-
-Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
-portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
-runtimes exist on Dankclaw).
-
-Evidence date: 2026-10-19. Everything below is grounded in:
-
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
-  (2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
-  `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
-  byte-for-byte from the live host.
- Upstream Qwen3-TTS model cards / release material.
-
-Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
-night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
-`libggml-cpu.so.0.27.0` and nothing else, and our own
-`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
-the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
-exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.
-
---
-
-## 1. What the payload is
-
-Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
-owned `dank:dank`, mtimes 2026-10-18 18:31), both from
-`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):
-
-| file | bytes | GGUF | tensors | arch string | file_type |
-|---|---|---|---|---|---|
-| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
-| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |
-
-`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
-to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
-files is unaccounted for.
-
-### 1.1 Talker file (480 tensors)
-
-Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
-(qwentts.cpp @ pinned commit) and confirmed by the tensor census:
-
-| prefix | tensors | bytes | what it is |
-|---|---|---|---|
-| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
-| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
-| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |
-
-Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
-`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.
-
-Which tensors carry which quant is the load-bearing portability fact:
-
- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
-  [2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
-  These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
-  embedding tables rather than to an `lm_head`.
- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
-  Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
-  `[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
-  `[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
-  GEMM shapes, nothing exotic.
- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
-  block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
-  `quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
-  the `text_proj` biases.
-
-Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
-`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
-`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
-the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
-think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
-portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
-special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
-the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
-repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
-**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
-eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.
-
-Two live-file details that contradict the upstream doc text (worth knowing when reading
-`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
-`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
-(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
-build shipped here has 480 because it carries `spk_enc.*`.
-
-### 1.2 Tokenizer/codec file (398 tensors)
-
-One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
-entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
-`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.
-
-Composition (metadata + tensor census, both files agree):
-
- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
-  conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
-  k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
-  LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
-  argmin loop. Metadata records `encoder.num_quantizers=32` with
-  `encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
-  the upstream config.
- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
-  `output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
-  head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
-  1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
-  transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
-  (`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
-  are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
-  → snake_post → conv_post k=7 → 96→1.
- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
-  deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
-  wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
-  quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
-  `quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
-  (7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
-  (Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
-  NPU question: every component is convolution, attention, or embedding lookup.
-
-### 1.3 Where the payload came from, and what the official weights look like
-
-Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):
-
-| repo | last modified | license | weights |
-|---|---|---|---|
-| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
-| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
-| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
-| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
-| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |
-
-So the GGUF is **not** the only public form: the safetensors are the canonical form
-(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
-them produced by the same author who wrote the runtime (`Serveurperso`), via
-`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
-Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
-(that count includes the embedded codec), the codec alone is 170.6 M params in F32.
-
-**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
-block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
-ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
-quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
-1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
-payload to a non-ggml stack therefore means starting from the safetensors again, not
-converting the GGUF.
-
-**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
-converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
-non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
-fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
-constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
-convs are not, at least not without re-measuring voice quality.
-
-## 2. What the runtime is
-
-`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
-application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
-`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
-range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
-build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.
-
-Five graph families, and their shapes are the whole portability story:
-
-| graph | batch/time axis | notes |
-|---|---|---|
-| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
-| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
-| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
-| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
-| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |
-
-Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
-audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
-fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
-batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
-chunked (buffered) codec decode.
-
-Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
-`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
-`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
-`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
-`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.
-
-**No custom ggml op is required today.** The repo doc still says the fork "provides the two
-custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
-pinned commit. Verified 2026-10-19:
-
- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
-  (`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
-  `include/ggml.h` is byte-identical to upstream master (`diff` empty);
- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
-  (mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
-  builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
-  and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
-  the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
-  (`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).
-
-Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
-payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.
-
-Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
-`CPU…`, default best available), resolved through the ggml backend registry; the deployed
-container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
-backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
-The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
-**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
-(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).
-
-Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
-empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
-`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
-(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
-mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
-(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
-(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.
-
-Parity expectations for any port: the upstream harness matches the PyTorch reference per
-stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
-reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
-one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
-accepted by listening + per-stage cosine, never by waveform diff against today's audio.
-
---
-
-## 3. The three answers Main asked for
-
-### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter
-
-There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
-container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
-GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
-op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
-official Apache-2.0 safetensors are upstream, and **four independent ports of the full
-pipeline already exist**:
-
-| port | form | what it proves |
-|---|---|---|
-| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
-| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
-| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
-| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |
-
-What the ports had to concede, and what any port inherits:
-
- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
-  support this generation structure yet", so their first shape was **three graphs + a
-  host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
-  loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
-  CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
-  0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
-  **warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
-  An NPU port is not "faster by default" for this workload; it has to beat that.
- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
-  in fp16/fp32 (the LiteRT split does exactly this).
-
-So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
-from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
-static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
-precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
-fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.
-
-### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS
-
-Checked against the trees today (2026-10-19), because this was the crux:
-
- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
-  `xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
-  HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
-  OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
-  matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
-  opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
-  2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
-  `GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
-  GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
-  (`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
-  token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
-  (HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
-  new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
-  unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
-  2,048-wide **lose** (0.77× at width 1,536).
- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
-  explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
-  amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
-  mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
-  int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
-  different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
-  12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
-  use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
-  conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
-  AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
-  picture of what XDNA2 graph execution costs.
- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
-  LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
-  `hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.
-
-Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
-by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
-sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
-covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
-elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
-this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
-graph on the NPU", which is exactly the NeuTTS-Air spike.
-
-### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP
-
- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
-  (Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
-  runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
-  `GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
-  OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
-  `--device Vulkan0` on the same command line the unit runs today.
- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
-  whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
-  `swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
-  `CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
-  WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
-  transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
-  runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
-  nothing in the tree is NVIDIA-specific.
- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
-  that the hardware/driver combination is viable: this very host already runs a Vulkan
-  endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
-  implementation is officially supported on Radeon Software for Linux
-  (AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
-  a build-and-measure spike, not an unsupported port.
- The real constraint is the one already in the fleet docs: the iGPU belongs to another
-  endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
-  with the text endpoints for the same silicon; the NPU route's only genuine advantage is
-  that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
-  is busy … no GPU contention").
-
---
-
-## 4. What any replacement must reproduce, and what the voices cost
-
-Endpoint contract in production today (from the live unit/env, not the doc):
-`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
-port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
-`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
-`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
-`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
-Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
-choice out of every consumer's config.
-
-Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
-(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
-`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
-silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
-artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
-`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
-re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
-registration time.
-
-That is the portable part, and it is the same shape AMD's NeuTTS flow wants
-(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
-*the WAV plus its transcript*, processed at request/registration time by the model's own
-speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
-inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
-`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
-replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
-re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
-that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
-(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
-(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
-ships a 1024-dim one).
-
-One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
-engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
-interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
-regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.
-
-## 5. Sources
-
-Runtime/payload (all read locally at pinned versions, 2026-10-19):
-
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
-  (commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
-  (860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
-  delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
-  https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
-  `include/ggml.h` identical to upstream
-  https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
-  (1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
-  the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
-  `scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.
-
-Model provenance:
-
- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
-  VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
-  2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
-  `speech_tokenizer/`) and 170,557,441 (F32 codec).
- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
-  `generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
-  `create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
-  flash-attn-2, Python 3.12 recommended).
- `https://github.com/QwenLM/Qwen3-TTS`.
-
-Portability precedents:
-
- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
-  `https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
-  `https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
-  `https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
-  `https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
-  `legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).
-
-AMD NPU / ggml backend status:
-
- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
-  XDNA/AIE/Vitis backend, 2026-10-19) ·
-  `https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
-  `https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
-  measured 2026-10-05/06 on HX 370) ·
-  `https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
-  phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
-  only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
-  **no TTS example anywhere in the official Ryzen AI Software tree**.
- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
-  models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
-  (Mesa Vulkan driver officially supported).
-
-Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
-portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
-which is flagged there.
@@ -1,191 +0,0 @@
-# TTS on the Strix Halo NPU — spike verdict (2026-10-19)
-
-Question: **how can the fleet's TTS engine run on hardware rather than CPU?** Answer in
-one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is
-the gfx1151 iGPU behind a build flag we already own.
-
-Spike evidence lives in `report/npu-tts-runtime-landscape-2026-10-19.md` (which runtimes can
-reach XDNA2 at all) and `report/qwen3-tts-payload-portability-2026-10-19.md` (what the
-payload is and where it can be moved). This page owns only the **measured spike result** and
-the **model-side contract** neither slice has. Throwaway work ran in containers under
-`/home/dank/spikes/npu-tts` on Dankclaw; nothing in the fleet was changed — `frank-tts` and
-`frank-tts-engine` still serve `narrator1|2|3` on CPU.
-
-Legend: **[host]** live census 2026-10-19 · **[spike]** measured in the spike containers ·
-**[vendor]** primary vendor source, with the version it holds for.
-
-## 1. Verdict
-
-**NO-GO on the NPU, and the wall is a loader.** The XDNA2 device is healthy by AMD's own
-instruments; the runtime that owns this model family opens the device and then declines the
-graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a
-path that cannot be reproduced is not a foundation. **[spike]**
-
-**Go on the iGPU, and the lever is in this repo.**
-`dankclaw/voiceservice/Containerfile.engine` builds `qwentts.cpp` (commit `5916bf7`) with
-`-DGGML_VULKAN=OFF` written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running
-container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-server` and
-nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
-the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
-world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
-gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
-the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
-budget while dragoon is loaded, so memory is not the constraint — which leaves the
-contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
-whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
-
-Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
-only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
-dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan
-exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.
-
-## 2. The two arms do not share a real-time bar
-
-|Arm|Talker|Codec|Codes per second of audio|Bar|
-|---|---|---|---|---|
-|**Deployed** (`frank-tts-engine`)|`qwen-talker-1.7b-base-Q4_K_M.gguf`|`qwen-tokenizer-12hz-Q8_0.gguf`|**12.5**|~13 tok/s **[host]**|
-|**Spike** (`amd/neutts-air_rai_1.8.0_npu_4K`)|NuTTS-Air, 24-layer Qwen2|NeuCodec `24000 / hop 480`|**50.00**|~50 tok/s **[spike]**|
-
-Measured from the staged NPU artifact — the contract any future arm must satisfy:
-
-|Fact|Value|
-|---|---|
-Vocab|`genai_config.json` `vocab_size` **217,652**; base BPE 151,643; specials 151,644–151,670 **[spike]**|
-Speech tokens|**65,536** `<|speech_N|>` entries, ids **151,671–217,206** contiguous; the vendor template maps a codec index straight to its id, `"".join(f"<|speech_{i}|>")`. A 10,001-code assumption silently truncates the space — encoded references really do reach the 60k range. **[spike]**|
-Stop token|`<|SPEECH_GENERATION_END|>` = 151,670, but the export's `model.eos_token_id` is `[151645, 151643]` (the *chat turn* stop) and `search.eos_token_ids` is unset — pass 151,670 explicitly or speech ends after one token **[spike]**|
-Input ceiling|4096 tokens **total**, input + output **[vendor]**|
-Frame rate|50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) **[spike]**|
-Reference encode|`librosa.load(sr=16000, mono=True)` → `[1,1,T]` float → `codec.encode_code(audio_or_path=…)` (vendor `NeuTTS.encode_reference`); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. `neucodec` also needs `torchtune`; install both `--no-deps --target <dir>` so a second torch never shadows the venv's. **[spike]**|
-Pre-encoded voices|`/spike/logs/codes-narrator{1,2,3}.npy` — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s **[spike]**|
-
-**Gap that any arm inherits:** the voice packs carry no transcript. `manifest.json` holds
-provenance and crop geometry (`crop.source_speech_run` 98.739–118.636 s of a 22,569 s
-audiobook) and no text, and Dankclaw has no ASR at all — `whisper`, `whisper.cpp`,
-`faster-whisper` all absent. NuTTS conditioning needs `ref_text`; do not invent it. **[host]**
-
-## 3. Exoneration chain — why the blame is not the hardware
-
-Every row is an instrument, not an inference.
-
-|Probe|Result|
-|---|---|
-`xrt-smi examine`|`[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8`, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, `amdxdna 7.0.0-38-generic` **[spike]**|
-`xrt-smi validate`|**PASSED**, including `throughput` **[spike]**|
-`pyxrt.device(0)` in-container|constructs — XRT enumerates the device **[spike]**|
-`strace -f -e trace=openat` on the failing load|`genai_config.json` read → `model.onnx` opened → `libxrt_driver_xdna.so.2` loaded → `/dev/accel/accel0` opened `O_RDWR` → GEM ioctls succeed → **EP declines, before touching `dd_metastate_*` or `cache/txn_bins.zip`** **[spike]**|
-`dmesg`, `fuser /dev/accel/accel0`|silent since boot, no holder — the decline never reaches the kernel **[spike]**|
-Hardlinked mirror dirs of the model tree (`rundir*`)|retired, no change — the "metastate shadowing" theory is dead **[spike]**|
-Every logging lever in that build: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT`|**zero lines** — the decline path is silent **[spike]**|
-Vendor `quicktest`|`Session successfully initialized` — proves the XRT/driver/device path, **not** NPU residency: that build falls back to CPU silently. Prove residency with `/sys/class/accel/accel0/device/mem_usage` during generation. **[spike] [vendor]**|
-
-No NPU token rate was ever measured, so none is quoted here. The single successful load
-(13:17, `[load] 0.84s`, finite logits, XRT banner at *generate* time) has not been
-reproduced by the byte-identical command since, with the model files pristine by inode and
-mtime. **[spike]**
-
-## 4. Host gates the spike proved (so they are not re-litigated)
-
- **Rootless containers cannot read `/dev/accel/accel0`** (`root:render` 0660): the host
-  gid never enters the userns map. `--group-add keep-groups`, `--userns=keep-id`,
-  `--userns=keep-id:gid=990`, `--group-add render`, `--group-add gid:990` — all fail.
-  Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet
-  already does for `kfd`/`renderD*`. **[spike]**
- **Never `dpkg -i` the vendor XRT deb** — it installs a DKMS `amdxdna` over the distro
-  driver. `dpkg-deb -x` the debs and bind-mount the tree at `/opt/xilinx/xrt`; the plugin
-  deb is what carries `libxrt_driver_xdna.so`. **[spike]**
- In-tree `amdxdna 0.7.0-38` + `iommu=pt` is enough for context creation; a missing device
-  node is *not* the modern failure mode. **[host]**
- `xrt-smi` inside a container needs `libboost-filesystem1.83.0` +
-  `libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`);
-  boost is needed by `xrt-smi` only, never by the OGA/EP path. **[spike]**
- Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census.
-  A multi-GB NPU partition plus an AIE compile workspace does not fit beside
-  `frank-dragoon`. Benign-looking `Failed to alloc dev bo memory, ret -28` is that
-  pressure. **[host]**
-
-## 5. Target shape: the TTS engine shares the gfx1151 with `frank-dragoon`
-
-The deployment Frank wants is **both resident at once**, not a handover. Measured 2026-10-19
-(read-only, nothing loaded or unloaded):
-
-|Measure|Value|Reading|
-|---|---|---|
-iGPU GTT in use|**18,523 MiB of 116,733 MiB**|≈96 GiB of aperture free; this is the real budget (fleet law) **[host]**|
-DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated — **not** the budget **[host]**|
-Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
-TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
-
-**Two gates, in this order.**
-
-1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
-   carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
-   `mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
-   `Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
-
-   |Heap|size|budget with dragoon loaded|flags|
-   |---|---|---|---|
-   |0|38.67 GiB|32.58 GiB|host-visible, not device-local|
-   |1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
-
-   The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
-   a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
-   of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
-   and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
-   Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
-   help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
-   ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
-   device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
-   3090 slot.
-2. **Contention gate — PASSED, measured 2026-10-19 after the cutover.** With a 16.9 s
-   synthesis occupying the device, `frank-dragoon` answered a streamed 124-token request at
-   **17 ms to first token and ~35 tok/s decode**, against 17 ms and ~33 tok/s for the same
-   request with the card idle. Compute-unit sharing showed up as nothing. One request per
-   arm, no ladder: these are the gate's evidence, not a throughput record, and the engine's
-   own shape lives in [[Frank TTS Engine]].
-
-**Change surface — as landed** (PR #34, `6248f02`; the CPU shape stays reachable as the
-rollback):
-
-1. `Containerfile.engine` build stage: add **`libvulkan-dev glslc spirv-headers`**. The
-   prediction above (`libvulkan-dev glslang-dev`) is wrong on two counts and the configure
-   says so: ggml looks for `glslc`, which Ubuntu's `glslang-tools` does **not** ship (it
-   ships `glslangValidator`), and `find_package(SPIRV-Headers CONFIG REQUIRED)` needs the
-   distro's `spirv-headers`, which does carry the CMake config. `glslc` is its own source
-   package on noble, at 2023.8 — old enough that only `GL_KHR_cooperative_matrix` compiles,
-   so the coopmat2 / integer-dot / bf16 / micro-float shader families are built out. Suspect
-   that compiler before the device if a Vulkan prefill ever reads as slow.
-2. `-DGGML_VULKAN=OFF` → `ON`: one word in the written-out flag list.
-3. Runtime stage: `mesa-vulkan-drivers` **and** `libvulkan1` — the ICD and its loader. An
-   image with a loader and no ICD enumerates zero devices.
-4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — the
-   DIRECTORY, never the node: on this hybrid box `renderD128` is the NVIDIA card and
-   `renderD129` the amdgpu iGPU (`readlink -f /sys/class/drm/renderD129/device/driver` →
-   `amdgpu`). On this host the udev rule already makes the nodes world-writable, so the
-   group is belt-and-braces, not the gate.
-5. Engine argv: `--device Vulkan0`, reached through `QWENTTS_DEVICE` in
-   `scripts/frank-tts-engine.env`. **An empty value passes no flag at all** — that is the
-   CPU rollback, and it is why the entrypoint's default is an `-v` test rather than `:=`,
-   which also fires on a set-but-empty value and would silently undo the rollback.
-
-Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
-throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
-the `VkPhysicalDeviceMemoryProperties` heap sizes — no model, no load, no compute. The probe
-trap: `vulkaninfo --batch` is not a flag in Ubuntu 24.04's `vulkan-tools` — it prints the help
-page and exits 1, which reads like a driver failure. Every package involved has a candidate on
-the host's apt (`vulkan-tools` 1.4.341, `libvulkan-dev` 1.4.341, `glslang-dev`/`glslang-tools`
-16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older
-but present. Script: `/home/dank/spikes/npu-tts/vk_probe.sh` on Dankclaw.
-
-Constraints that do not relax for a co-tenant: `/dev/dri` passthrough needs the ComfyUI
-pattern (`AddDevice` + `GroupAdd=keep-groups` + the world-writable udev rule) because
-systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at `/swap.img`
-exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents;
-`pp_dpm_fclk` reads `auto` (unheld), which only becomes a hazard if the NPU ever joins.
-
-## 6. Re-visit condition
-
-Retry the NPU when **a Linux loader claims the graph**: an `onnxruntime-genai-ryzenai`
-build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on `RyzenAI-npu5`, or that
-model appearing in a runtime's own supported catalog with a Linux runner. Then the
-prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices,
-the `ref_text` transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.
@@ -0,0 +1,309 @@
+#!/usr/bin/env python3
+"""bench-spec-ab.py — settle "which proposer should this endpoint use?" with data.
+
+Runs one payload through several speculative-decoding configurations of the same
+binary, one at a time, with the unit stopped between runs: on a single-GPU host
+there is no room for two copies of a 29 GiB model, and the benchmarking policy
+applies. Every configuration gets identical prompts at temperature 0, and the
+greedy text is compared across them — speculation may buy speed, never change
+the answer.
+
+    bench-spec-ab.py <env-file> <serve-script> [options]
+    bench-spec-ab.py /home/dank/serverAppConfigs/scripts/frank-medium.env \
+                     /home/dank/serverAppConfigs/scripts/serve-frank-medium.sh \
+                     --configs "dflash=draft-dflash@7;ngram=ngram-mod@64;chain=draft-dflash,ngram-mod@7"
+
+Config syntax: `name=SPEC_TYPE[@n_max][|ENV=VAL|ENV=VAL]`, entries joined by `;`.
+The systemd unit is restored in a finally block, so it comes back on whatever the
+env file says even if a configuration crashes the harness.
+
+THREE SHAPES, because a single one answers the wrong question. A proposer can
+only draft text it has seen, so the shape decides the verdict:
+
+  prose       a 1k prompt asking for one sentence. Nothing to copy and nothing
+              repetitive, so only a model-backed proposer can help: this is the
+              shape that compares drafters.
+  echo        a ~1.5k passage of unique lines, "repeat it back verbatim". The
+              output IS the prompt, which is ngram-mod's home turf.
+  structured  "25 JSON records, one per line" — repetitive OUTPUT with no prompt
+              overlap, which is what an ngram matcher is really for in an
+              agentic fleet (config edits, tables, record lists).
+
+echo and structured disable thinking (`chat_template_kwargs`), otherwise the
+model thinks the answer into `reasoning_content`, `content` comes back empty and
+the cross-configuration identity check passes vacuously — which is exactly how
+this harness first "verified" an echo it never looked at.
+
+Results on Dankserver (frank-medium payload, 2026-09-16, temperature 0). Decode:
+
+  shape            dflash=7   ngram-mod(48/64)   chain        (tok/s)
+  prose 1k/512        43.8        17.7 (0 drafts)   43.7
+  echo 1.5k/400       76.8       102.5             200.6
+  structured/400      91.1        17.8 (0 drafts)   91.2
+  identity            70.4        17.8 (0 drafts)   70.4
+
+`draft-mtp,draft-dflash` fails at load (a chain has ONE draft context and MTP
+builds against whatever model that context holds); ngram-mod has no such
+constraint — it is draftless and joins the chain unconditionally.
+"""
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+import time
+import urllib.request
+
+FILLER = ("The quick brown fox jumps over the lazy dog near the river bank. "
+          "A cache is a component that stores data so future requests are faster. ")
+TOKENS_PER_REP = 28
+UNIT = "frank-medium.service"
+MODEL = "frank-medium"
+SPEC_GREP = ("adding speculative implementation|- n_max=|- block_size=|creating MTP|"
+             "draft acceptance")
+
+DEFAULT_CONFIGS = [
+    ("none", {"SPEC_TYPE": "none"}),
+    ("mtp-n3", {"SPEC_TYPE": "draft-mtp", "SPEC_DRAFT_N_MAX": "3"}),
+    ("mtp-n7", {"SPEC_TYPE": "draft-mtp", "SPEC_DRAFT_N_MAX": "7"}),
+    ("dflash-n7", {"SPEC_TYPE": "draft-dflash", "SPEC_DRAFT_N_MAX": "7"}),
+]
+
+WORDS = ["latency", "throughput", "cache", "adapter", "scheduler", "allocator",
+         "queue", "kernel", "buffer", "shard", "replica", "window"]
+
+
+def echo_passage(lines=90):
+    """Unique lines: no long span repeats inside the prompt, so an n-gram can
+    only fire when the OUTPUT copies the INPUT."""
+    return "\n".join(
+        "Record %03d: the %s of subsystem %d reports state %d with checksum %05d."
+        % (i, WORDS[i % len(WORDS)], i % 7, i % 3, (i * 7919) % 100000)
+        for i in range(lines))
+
+
+ECHO = (echo_passage() + "\n\nRepeat the passage above verbatim, every line, exactly as "
+        "written, with no commentary before or after it.")
+STRUCTURED = ("Output exactly 25 JSON records, one per line, no prose and no code fence. "
+              'Each line must look like {"id": N, "state": "active", "region": "us-west-2"} '
+              "with N running from 1 to 25.")
+PROSE = FILLER * (1000 // TOKENS_PER_REP) + "\n\nReply with a single short sentence."
+IDENTITY = "List the first five prime numbers, comma separated, then stop."
+NO_THINK = {"chat_template_kwargs": {"enable_thinking": False}}
+
+
+def log(msg, logfile):
+    line = "[" + time.strftime("%H:%M:%S") + "] " + str(msg)
+    print(line, flush=True)
+    with open(logfile, "a") as fh:
+        fh.write(line + "\n")
+
+
+def sh(cmd):
+    return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout.strip()
+
+
+def parse_configs(spec):
+    """`name=SPEC_TYPE[@n_max][|KEY=VALUE]...`, entries joined by ';'.
+
+    KEY=VALUE parts override any env file key, which is how an impl's own window
+    (SPEC_NGRAM_N_MIN/MAX, for instance) gets swept without hand-editing env
+    files."""
+    out = []
+    for chunk in spec.split(";"):
+        chunk = chunk.strip()
+        if not chunk:
+            continue
+        name, _, rest = chunk.partition("=")
+        over = {}
+        if "|" in rest:
+            rest, _, extras = rest.partition("|")
+            for item in extras.split("|"):
+                if "=" in item:
+                    k, _, v = item.partition("=")
+                    over[k.strip()] = v.strip()
+        n_max = None
+        if "@" in rest:
+            rest, _, n_max = rest.partition("@")
+        over["SPEC_TYPE"] = rest.strip()
+        if n_max:
+            over["SPEC_DRAFT_N_MAX"] = n_max.strip()
+        out.append((name.strip(), over))
+    return out
+
+
+def write_env(src, name, overrides):
+    """The env file with this configuration's overrides applied."""
+    path = "/tmp/spec-ab-" + name.replace("/", "_") + ".env"
+    with open(src) as fh:
+        lines = fh.read().splitlines()
+    out = []
+    for line in lines:
+        for k, v in overrides.items():
+            if line.startswith(k + "="):
+                line = k + "=" + v
+                break
+        out.append(line)
+    with open(path, "w") as fh:
+        fh.write("\n".join(out) + "\n")
+    return path
+
+
+def post(base, body, timeout=1800):
+    req = urllib.request.Request(
+        base + "/v1/chat/completions", data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.load(r)
+
+
+def vram():
+    out = sh("rocm-smi --showmeminfo vram 2>/dev/null | grep 'Used Memory'")
+    return [round(int(x) / 1e9, 2) for x in re.findall(r":\s+(\d+)\s*$", out, re.M)][:2]
+
+
+def wait_health(base, timeout=420):
+    t0 = time.time()
+    while time.time() - t0 < timeout:
+        if not sh("pgrep -x llama-server >/dev/null && echo up"):
+            return False
+        try:
+            with urllib.request.urlopen(base + "/health", timeout=5) as r:
+                if r.status == 200 and json.load(r).get("status") == "ok":
+                    return True
+        except Exception:
+            time.sleep(3)
+    return False
+
+
+def stop_unit():
+    sh("sudo systemctl stop " + UNIT)
+    sh("pkill -x llama-server")
+    for _ in range(40):
+        if not sh("pgrep -x llama-server || true"):
+            return
+        time.sleep(1)
+
+
+def restore_unit(base, logfile):
+    """Bring the endpoint back on the deployed configuration. Runs in a
+    finally, because a harness that can leave production down is a hazard."""
+    log("restoring " + UNIT, logfile)
+    sh("sudo systemctl start " + UNIT)
+    for _ in range(120):
+        try:
+            with urllib.request.urlopen(base + "/health", timeout=5) as r:
+                if json.load(r).get("status") == "ok":
+                    log(UNIT + " healthy again", logfile)
+                    return True
+        except Exception:
+            time.sleep(5)
+    log("WARNING: " + UNIT + " did not come back healthy -- check it by hand", logfile)
+    return False
+
+
+def run_shape(base, prompt, max_tokens, label, extra=None):
+    body = {"model": MODEL, "messages": [{"role": "user", "content": prompt}],
+            "max_tokens": max_tokens, "temperature": 0.0}
+    if extra:
+        body.update(extra)
+    d = post(base, body)
+    tm = d["timings"]
+    msg = d["choices"][0]["message"]
+    content = (msg.get("content") or "").strip()
+    row = {"shape": label,
+           "prompt_n": tm["prompt_n"],
+           "prefill_tps": round(1000 * tm["prompt_n"] / tm["prompt_ms"], 1),
+           "predicted_n": tm["predicted_n"],
+           "decode_tps": round(tm["predicted_per_second"], 2),
+           "draft_n": tm.get("draft_n", 0),
+           "accepted": tm.get("draft_n_accepted", 0),
+           "answer": content,
+           "reasoning_len": len(msg.get("reasoning_content") or "")}
+    if label.startswith("echo"):
+        # How much of the answer really is the passage: a configuration that
+        # "wins" by answering something else is not a win.
+        want = {l.strip() for l in echo_passage().splitlines()}
+        got = [l.strip() for l in content.splitlines() if l.strip()]
+        row["copy_ratio"] = round(sum(1 for l in got if l in want) / len(got), 3) if got else 0.0
+    return row
+
+
+def measure(base):
+    run_shape(base, FILLER * (4000 // TOKENS_PER_REP) +
+              "\n\nReply with a single short sentence.", 32, "warmup")
+    return [
+        run_shape(base, PROSE, 512, "prose 1k/512"),
+        run_shape(base, ECHO, 400, "echo 1.5k/400", NO_THINK),
+        run_shape(base, STRUCTURED, 400, "structured/400", NO_THINK),
+        run_shape(base, IDENTITY, 192, "identity"),
+    ]
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("env_file")
+    ap.add_argument("serve_script")
+    ap.add_argument("--base-url", default="http://127.0.0.1:7770")
+    ap.add_argument("--log", default="/tmp/bench-spec-ab.log")
+    ap.add_argument("--configs", default=None,
+                    help="name=SPEC_TYPE[@n_max][|ENV=VAL] joined by ';' "
+                         "(default: the four configs measured on 2026-09-16)")
+    ap.add_argument("--json", default="/tmp/bench-spec-ab.json")
+    args = ap.parse_args()
+
+    configs = parse_configs(args.configs) if args.configs else DEFAULT_CONFIGS
+    # Absolute paths: the launcher must not depend on the caller's cwd, and a
+    # relative path plus a `cd` to its own directory double-counts the prefix.
+    env_file = os.path.abspath(args.env_file)
+    serve_script = os.path.abspath(args.serve_script)
+    base, logfile = args.base_url, args.log
+    results = {}
+    log("spec A/B start: " + ", ".join(n for n, _ in configs) +
+        " (unit " + UNIT + " stopped for the duration)", logfile)
+    try:
+        stop_unit()
+        for name, over in configs:
+            env_path = write_env(env_file, name, over)
+            logf = "/tmp/spec-ab-" + name.replace("/", "_") + ".log"
+            sh("FRANK_MEDIUM_ENV_FILE=" + env_path + " setsid bash " + serve_script +
+               " > " + logf + " 2>&1 < /dev/null &")
+            if not wait_health(base):
+                log(name + ": FAILED to become healthy -- " + sh("tail -3 " + logf), logfile)
+                results[name] = {"error": "not healthy", "tail": sh("tail -6 " + logf)}
+                stop_unit()
+                continue
+            rows = measure(base)
+            results[name] = {"vram_gb": vram(), "rows": rows,
+                             "spec_log": sh("grep -E '" + SPEC_GREP + "' " + logf + " | tail -4")}
+            for r in rows:
+                log(name + " | " + r["shape"] + " prefill " + str(r["prefill_tps"]) +
+                    " tok/s | decode " + str(r["decode_tps"]) + " tok/s | draft " +
+                    str(r["draft_n"]) + "/" + str(r["accepted"]) +
+                    (" (" + str(round(100.0 * r["accepted"] / r["draft_n"])) + " %)"
+                     if r["draft_n"] else "") +
+                    (" | copy " + str(r["copy_ratio"]) if "copy_ratio" in r else "") +
+                    (" | EMPTY ANSWER" if not r["answer"] else ""), logfile)
+            log(name + " | vram " + str(results[name]["vram_gb"]), logfile)
+            stop_unit()
+    finally:
+        restore_unit(base, logfile)
+
+    with open(args.json, "w") as fh:
+        json.dump(results, fh, indent=2)
+    for shape in ("prose 1k/512", "echo 1.5k/400", "structured/400", "identity"):
+        texts = [r["answer"] for v in results.values() if "rows" in v
+                 for r in v["rows"] if r["shape"] == shape]
+        if any(not t for t in texts):
+            log("WARNING: '" + shape + "' produced an empty answer in some configuration -- "
+                "its identity check proves nothing", logfile)
+        elif len(set(texts)) > 1:
+            log("WARNING: configurations disagree on greedy text for '" + shape +
+                "' -- speculation is changing answers", logfile)
+    log("done; results in " + args.json, logfile)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,318 @@
+# Tunables for frank-medium (orcarouter/Qwen3.8-27B-Uncensored-GGUF Q8_0 +
+# z-lab/Qwen3.8-27B-DFlash2-GGUF Q4_K_M drafter), mainline llama.cpp on
+# Dankserver, 2x AMD Radeon AI PRO R9700 (gfx1201).  Since 2026-09-16 this is
+# the SOLE LLM runtime on the box: the vllm-radiance replica pair and its
+# routing proxy are retired.
+#
+# Hardware: two R9700 (Navi 48, RDNA4, 32,624 MiB each), no XGMI, different
+# IOMMU groups -- every cross-card byte goes over PCIe.  The Raphael iGPU
+# (gfx1036/Radeon 610M) enumerates as HIP device 2 and is hidden below so
+# nothing can drift onto shared system RAM.
+#
+# WHY LAYER SPLIT AND NOT TENSOR (operator call, 2026-09-16): tensor parallel
+# needs a cross-card allreduce per layer, and this host cannot dispatch one.
+# RCCL's kernels carry a hidden hostcall-buffer parameter, and ROCr refuses it
+# on a device without PCIe atomics -- the second card sits behind the B650
+# chipset bridges (AtomicOpsCap: Routing-, read-only), so every collective that
+# touches it dies with hipErrorIllegalState.  Measured on this box 2026-09-15
+# (llama-bench, pp4096 / tg128, f16 KV, fa on):
+#
+#   single R9700    1160.9 / 19.42
+#   layer  2x       1991.2 / 18.76      (859.1 / 17.64 at depth 32768)
+#   tensor 2x        271.8 / 26.26      (no RCCL: butterfly fallback, -7x prefill)
+#
+# Layer split distributes layers in order; only the activation at the boundary
+# crosses the wire, once per forward pass.  That is the "pipeline parallelism"
+# this service wanted, and it needs no collective at all.
+#
+# Runtime: mainline llama.cpp master fb27a525 (2026-09-16), gfx1201, built by
+# scripts/build-llama-rdna4.sh against a private ROCm 7.14.0a20260624 toolchain
+# that is bundled next to the binary (RPATH $ORIGIN/../lib, no LD_LIBRARY_PATH).
+# The ref is pinned by TWO features: DFlash2 (ggml-org/llama.cpp#27342 -- the
+# drafter is detected from its GGUF) and qwen35 MTP.  Do not float it.
+#
+# Model: orcarouter/Qwen3.8-27B-Uncensored-GGUF @ Q8_0, 27.0 GiB, arch qwen35
+# (verified from the GGUF header 2026-09-19: 262144 trained window, 24 heads /
+# 4 KV heads, nextn_predict_layers=1).  The file ALSO carries the model's MTP
+# head (blk.64.nextn.*), so --spec-type draft-mtp is available without a second
+# copy in VRAM -- one SPEC_TYPE change, the drafter below stays on disk.
+#
+# Drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF @ Q4_K_M, 1.06 GiB (operator call
+# 2026-09-19: the Q4 rung -- the drafter's job is throughput, not output
+# quality, and the target verifies every drafted token).  5 layers, hidden
+# 5120, a block-diffusion proposal (block_size 8) under a 2048-token sliding
+# window (dflash.attention.sliding_window=2048, verified from the header).
+# The window is what makes it cheap HERE: its KV is bounded at 2048 positions
+# (~40 MiB) instead of spanning the context (see SIZING).
+#
+# Architecture (GGUF arch qwen35): 64 layers as 16 x (3 x (Gated DeltaNet ->
+# FFN) + 1 x (Gated Attention -> FFN)) -- 48 linear-attention layers whose
+# per-sequence state is CONSTANT-SIZE, 16 full-attention layers at 4 KV heads x
+# 256 head_dim.  A token therefore costs 2 x 4 x 256 x 2 B = 64 KiB of KV at f16
+# in the attention layers only, plus ~0.16 GiB per sequence of f32 DeltaNet
+# state at any depth.
+#
+# SIZING, the deployed shape (32:33 / ctx 327680 / 6144x512 / two slots /
+# DFlash2 Q4_K_M), measured 2026-09-19 after the model swap (rocm-smi):
+#
+#   card 0 (x16)  26.9 GiB used, 5.7 GiB free
+#   card 1 (x1)   27.8 GiB used, 4.8 GiB free
+#
+# The predecessor's measured table (rocm-smi, 2026-09-16, UD-Q8_K_XL target)
+# stays as the calibration history:
+#
+#   CTX     split  batch  slots  drafter  GPU 0 free  GPU 1 free   note (ubatch 1536)
+#   262144   1,1   2048   1      DFlash2   8.64 GiB    4.43 GiB   day's start
+#   262144  39,26  6144   2      DFlash2   2.42 GiB    7.20 GiB
+#   262144  38,27  6144   2      DFlash2   3.85 GiB    5.77 GiB
+#   262144  37,28  6144   2      DFlash2   4.28 GiB    5.34 GiB
+#   262144  36,29  6144   2      DFlash2   4.71 GiB    4.91 GiB
+#   262144  35,30  6144   2      DFlash2   5.14 GiB    4.48 GiB
+#   262144  40,25  6144   2      DSpark    1.87 GiB    1.86 GiB   (retired)
+#
+# ...and the same deployed shape re-measured at ubatch 512, after llama-bench
+# showed 512 is the throughput optimum and costs 3.71 GiB less in total (see the
+# batch block below):
+#
+#   262144  36,29  6144   512    2      DFlash2   6.01 GiB    7.32 GiB   ubatch only
+#   262144  35,30  6144   512    2      DFlash2   6.44 GiB    6.90 GiB
+#
+# ONE -ts STEP IS ONE LAYER, AND LAYERS ARE NOT THE SAME SIZE.  This is a hybrid
+# stack -- 16 x (3 Gated DeltaNet + 1 Gated Attention) blocks -- so moving a
+# DeltaNet layer shifts ~0.43 GiB while an attention layer shifts ~1.43 GiB
+# (0.45 of weights plus ~1.0 GiB of its own f16 KV at 262144).  Both step sizes
+# were measured, which is why the balance is found by measurement rather than by
+# interpolating between splits.
+#
+# WHY THE SPLIT IS 32:33 (operator call, 2026-09-19).  The operator asked for
+# the tensors BALANCED across the two cards, which for 65 layers is 32:33 --
+# the most even split that exists.  The predecessor's 35:30 skew existed to lean
+# work onto the x16 card (ROCm0), because ROCm1's Gen4 x1 chipset path made it
+# the 98 % utilization bottleneck; the measured trade of the balanced point is
+# that the x1 card runs hotter while the x16 card idles a little more, and the
+# operator accepted that for symmetry.  Do not re-skew without asking.
+#
+# THE TWO CARDS' PATHS ARE NOT SYMMETRIC (measured, 2026-09-17): ROCm0
+# (03:00.0) hangs off CPU root port 00:01.1 at Gen4 x16 end to end (~31.5 GB/s);
+# ROCm1 (0b:00.0) rides the B650 chipset through bridges that each negotiate
+# Gen4 x1 (~1.97 GB/s).  The layer-split boundary itself is not the cost
+# (~5.2 MB of activation per 512-token ubatch rides the wire); the x1 card is
+# COMPUTE-bound, not PCIe-bound.
+#
+# A FAULT TO WATCH.  A 39:26 split, at 1x batch / 1 slot, died on its eighth
+# request: the amdgpu driver logged SDMA0 UTCL2 page faults and the engine
+# surfaced `ROCm error: an illegal memory access was encountered` -> ggml_abort,
+# i.e. a fault in the cross-card boundary copy, this layer split's hot path.  It
+# has not reproduced since, but the signature is worth recognising -- it is NOT
+# an OOM, and no amount of headroom prevents it.
+#
+# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
+# those bare vars; 2026-08-13 incident).  llama.cpp takes --host/--port/--temp
+# as flags only.
+
+# llama-server: mainline fb27a525 built for gfx1201 by scripts/build-llama-rdna4.sh.
+LLAMA_BIN=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server
+
+# Payload, fetched with scripts/download-frank-medium-gguf.sh.
+MODEL_DIR=/home/dank/models/Qwen3.8-27B-Uncensored-GGUF
+MODEL_FILE=Qwen3.8-27B-Uncensored-Q8_0.gguf
+# The f16 vision projector, same repo as the target.  llama-server loads it via
+# the explicit --mmproj flag (no autoload in this build); the serve script
+# refuses to start without it when vision is enabled, so a missing file is a
+# deploy error, not a text-only server that pretends to be multimodal.
+MM_PROJ_FILE=mmproj-Qwen3.8-27B-Uncensored-f16.gguf
+DRAFT_MODEL_DIR=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF
+DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q4_K_M.gguf
+
+# Fleet identity.  7770 is the fleet's dankserver endpoint; the alias is the id
+# at /v1/models and what every client asks for.
+ALIAS=frank-medium
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770
+# Device placement.  HIP_VISIBLE_DEVICES hides the Raphael iGPU (HIP device 2)
+# so nothing can drift onto shared system RAM; DEVICES is what llama.cpp splits
+# across, and neither of the two cards has an XGMI link to the other.
+HIP_VISIBLE_DEVICES=0,1
+DEVICES=ROCm0,ROCm1
+SPLIT_MODE=layer
+
+# 32:33 -- balanced (operator call 2026-09-19): 65 layers split as evenly as
+# they can be, ROCm1 (the last-layers card, which also carries the drafter and
+# the output layers) holding the extra one.  See the WHY THE SPLIT IS 32:33
+# block above for the trade against the predecessor's 35:30 x16-lean.
+TENSOR_SPLIT=32,33
+
+NGL=999
+# Auto-fit has no business resizing a context we set explicitly.
+FIT=off
+FLASH_ATTN=on
+
+# 327680 positions over TWO slots, so each conversation gets 163840 --
+# --parallel DIVIDES --ctx-size, so more slots would shrink every
+# conversation rather than grow the cache.  163840 is inside the checkpoint's
+# trained 262144 window, so the whole pool is trained territory (no rope
+# extrapolation, unlike the predecessor's 393216).
+CTX_SIZE=327680
+PARALLEL=2
+
+# MEASURED 2026-09-16 with scripts/bench-batch-ubatch.sh on this box: llama-bench,
+# a 64k prefill plus a 128-token decode with 64k of KV already resident, 3 reps,
+# one invocation per cell, the deployed split and f16 KV.  --ubatch-size is the
+# physical batch the graph is built at -- and the only one of the two knobs that
+# costs VRAM:
+#
+#   batch  ubatch   prefill tok/s        GPU 0 / GPU 1 free
+#   6144   1536    1164.96 +- 2.30      10.28 / 11.71 GiB
+#   6144    768    1135.03 +- 1.09      11.29 / 13.28 GiB
+#   6144    512    1192.93 +- 0.91      11.63 / 13.81 GiB   <- deployed
+#   6144    384    1068.04 +- 0.79      11.80 / 14.07 GiB
+#   6144    256    1045.98 +- 1.69      11.97 / 14.33 GiB
+#   4096   1536    1161.90 +- 0.96      10.28 / 11.71 GiB
+#   2048   1536    1170.23 +- 0.86      10.28 / 11.71 GiB
+#   2048    512    1193.88 +- 1.01      11.63 / 13.81 GiB
+#
+# THE PLATEAU STARTS AT 512.  512 -> 1536 buys ~2 % of prefill -- inside the
+# run-to-run spread, and the 512 cells were the faster pair in BOTH batch
+# columns -- while costing 1.35 GiB on ROCm0 and 2.10 GiB on ROCm1, so the
+# deployment takes the cheap end.  BELOW 512 the SMs are no longer saturated:
+# 384 and 256 fall ~11 % and ~12 %.  The logical batch is free either way --
+# 2048, 4096 and 6144 land within 2 % of each other at a fixed ubatch and use
+# the SAME VRAM, because the compute buffers are sized by ubatch alone.
+#
+# The prefill figure is a property of the SHAPE, not of this knob: the same
+# sweep with no depth in front of it (depth 0) runs 1543 tok/s at ubatch 256 and
+# 1723 at 1536 -- the plateau is in the same place, and the ~32 % between the two
+# tables is what attending over 64k of resident KV costs.  Decode is 15.88 tok/s
+# at depth 65536 on EVERY ubatch (17.80 at depth 0), so this knob cannot touch
+# generation at all.
+#
+# 8192/2048 stays untested under this drafter (the "2020 MiB pp buffer fails on
+# ROCm1" note was measured under DSpark, when it had 0.08 GiB free), and there is
+# no longer a reason to want it: batch buys nothing measurable here.
+BATCH_SIZE=6144
+UBATCH_SIZE=512
+
+# Threads cover tokenization, sampling and HTTP only: every layer is on a GPU.
+NUM_THREADS=8
+THREAD_BATCH=8
+
+# f16 K/V: an agentic checkpoint is the wrong place to spend KV precision.
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+# Context checkpoints: 0 -- no per-slot KV snapshots at all.  The build's
+# default is 32 (this binary's --help), so the 0 below is a deliberate
+# deviation, not an inherited default: a checkpoint is a VRAM->host-RAM copy
+# of part of a slot's KV, taken so a partially-matching prompt can be restored
+# instead of re-prefilled.  This endpoint trades that away -- its reuse comes
+# from the slot prompt and from the RAM-side prompt cache below.  Stated
+# explicitly so the tunable stays visible rather than merely absent.
+CTX_CHECKPOINTS=0
+# PROMPT CACHE, HOST RAM -- this is the one cache tunable that is NOT VRAM.
+# --cache-ram caps the total bytes of saved prompt states
+# (server_prompt_cache, tools/server/server-task.cpp): update() evicts
+# oldest-first past the cap, and alloc() REFUSES outright any single state
+# bigger than the whole cap.  A state is the serialized KV of a whole sequence
+# plus its DeltaNet state plus the draft context's own KV, so at this shape it
+# costs ~68 KiB per token --
+#   16 full-attention layers x 4 kv heads x 256 dim x 2 B x 2 (K,V) = 64 KiB
+#   plus the draft context's window-bounded KV (a ~40 MiB constant for DFlash2)
+# -- which makes the cap a TOKEN budget in disguise: 16384 MiB holds ~240k
+# tokens, i.e. a FULL 163840-token slot (~10.6 GiB) with ~5 GiB of the budget
+# spare for a second entry.
+#
+# The cost is host RAM, allocated on demand and never pinned (plain heap
+# vectors), so no VRAM number above moves.  dankserver has 62 GiB beside the
+# media-stack co-tenants, so the cache is bounded rather than unlimited (-1):
+# the seam test refuses a budget over half the host's RAM.
+CACHE_RAM=16384
+
+# Sampling defaults for requests that send none -- the values the vLLM
+# predecessor served on this endpoint (`--override-generation-config`), so a
+# client sees no change across the runtime swap.  The model card's own
+# recommendation is temp 1.0 / top_p 0.95; the harness sets its own per model in
+# models.yml, so this is the floor, not the contract.
+TEMPERATURE=0.7
+TOP_P=0.95
+TOP_K=20
+MIN_P=0.0
+PRESENCE_PENALTY=0.0
+
+# Reasoning on, projected into reasoning_content (deepseek format) with no
+# budget: the model decides how long to think.
+REASONING=on
+REASONING_FORMAT=deepseek
+REASONING_BUDGET=-1
+
+# Chat template: empty = the one embedded in the GGUF (Qwen3.8's own).
+CHAT_TEMPLATE_FILE=
+CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
+
+# Speculative decoding: the DFlash2 sidecar (operator call 2026-09-19).  The
+# target file also carries its own MTP head (nextn_predict_layers=1), so
+# draft-mtp is one SPEC_TYPE change away -- the drafter file stays on disk.
+# n_max 7 is the measured DFlash2 winner (see the table below); the drafter
+# rides ROCm1, the card that holds the last layers.
+SPEC_TYPE=draft-dflash
+SPEC_DRAFT_N_MAX=7
+SPEC_DRAFT_NGL=999
+SPEC_DRAFT_DEVICE=ROCm1
+SPEC_DRAFT_TYPE_K=f16
+SPEC_DRAFT_TYPE_V=f16
+
+# DFlash2 vs the model's own MTP head -- MEASURED 2026-09-16 on the predecessor
+# payload (temperature 0, 1k prompt / 512 out; the autoregressive baseline and
+# both MTP depths were run in one session, same binary, same prompts):
+#
+#   none            17.7 tok/s                     card1 26.07 GB
+#   draft-mtp  n=3  32.4 tok/s  58 % accepted       card1 29.35 GB
+#   draft-mtp  n=7  26.7 tok/s  33 % accepted       card1 29.65 GB
+#   draft-dflash 7  43.5 tok/s  41 % accepted       card1 29.42 GB
+#
+# All four produced IDENTICAL greedy text, so verification is lossless
+# whichever proposer runs.  DFlash2 is the faster proposer (2.46x over
+# autoregressive, 34 % over the best MTP).  The MTP head adds ~3.3 GB to the
+# card that holds it, against 1.06 GB for the Q4_K_M drafter's file -- at the
+# 327680 pool the drafter's window-bounded KV (~40 MiB) leaves headroom on
+# both cards, so the speed wins.
+#
+# THE TWO CANNOT BE CHAINED. `--spec-type draft-mtp,draft-dflash` dies at load:
+#   context type MTP requested but model doesn't contain MTP layers
+#   failed to create MTP context
+# A chain has ONE draft context, and MTP builds against whatever model that
+# context holds -- here the DFlash2 file, which has no MTP layers. MTP's head is
+# only loaded when MTP IS the draft model ("creating MTP draft context against
+# the target model ..."), so a chain of the two is a structural dead end, not a
+# tuning question. See serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md.
+
+# ngram-mod: the draftless n-gram matcher (CPU hash table in host RAM, no VRAM,
+# no model file, no draft context).  It joins the chain unconditionally --
+# unlike MTP, which needs a draft context and therefore cannot share one with
+# DFlash2.  NOT CURRENTLY ACTIVE: SPEC_TYPE is draft-dflash alone, and the serve
+# script only passes --spec-ngram-mod-* when the type contains "ngram".  The
+# window below is retained because the DFlash2 + ngram chain was measured free
+# on prose and 2.6x on prompt-echo -- one SPEC_TYPE change brings it back.
+#
+# WINDOW MEASURED 2026-09-16 (chain at each window, decode tok/s):
+#
+#   n_min/n_max    prose 1k/512   prompt-echo 1.5k/400
+#   48/64          43.9           200.6
+#   16/32          36.0           161.0
+#    8/16          36.2           113.8
+#
+# The long window is the right one, and not only because it wins on echo: a
+# SHORT window makes the matcher propose on short fragments of ordinary prose,
+# where the proposals are rejected and the verification still costs -- which is
+# the "ngram hurts" result the fleet measured on other payloads twice before.
+# With 48/64 the chain is byte-identical to DFlash2 alone on prose (same 119
+# drafts / 49 accepted) and on the identity prompt (70/53).
+#
+# Do not shorten these without re-running scripts/bench-spec-ab.py: the
+# no-penalty property is what makes the chain safe as a default.
+SPEC_NGRAM_N_MATCH=24
+SPEC_NGRAM_N_MIN=48
+SPEC_NGRAM_N_MAX=64
+
+# Vision enabled: llama-server loads mmproj-Qwen3.8-27B-Uncensored-f16.gguf
+# (931 MiB, lands on ROCm0 -- the fit charges it there) via the explicit
+# --mmproj flag.  This keeps frank-medium multimodal across the model swap.
+NO_MMPROJ=0
@@ -0,0 +1,20 @@
+[Unit]
+Description=llama.cpp Qwen3.8-27B Uncensored UD-Q8_K_XL + DFlash2 drafter (dankserver, frank-medium, 2x Radeon AI PRO R9700)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+# The engine pins staging buffers on the first card; keep the memlock budget
+# unbounded so an RLIMIT that is smaller than the split buffers cannot fail the
+# load.  Parity with the other LLM units on this fleet.
+LimitMEMLOCK=infinity
+Environment="FRANK_MEDIUM_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-medium.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-medium.sh
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,203 @@
+#!/usr/bin/env bash
+# Serve ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF @ UD-Q8_K_XL as
+# frank-medium on Dankserver's two Radeon AI PRO R9700 (gfx1201), with the
+# z-lab DFlash2 block-diffusion drafter, using mainline llama.cpp.
+#
+# Runtime: llama.cpp master fb27a525 (2026-09-16), a gfx1201 HIP build that
+# bundles its own ROCm 7.14 runtime (RPATH $ORIGIN/../lib), produced by
+# scripts/build-llama-rdna4.sh.  Two features pin that ref: DFlash2
+# (ggml-org/llama.cpp#27342) and qwen35 MTP.  The charlie12345/ROCmFPX fork
+# this deployment replaced has neither (its tip is c49ebdb, 2026-08-22).
+#
+# Parallelism: --split-mode layer across both cards.  Tensor split would need a
+# cross-card allreduce and this host cannot dispatch one -- RCCL's kernels
+# require a hostcall buffer that ROCr refuses on a card without PCIe atomics,
+# and the second R9700 sits behind the chipset (AtomicOpsCap: Routing-).  Layer
+# split crosses the wire once per forward pass instead of once per layer.
+#
+# The vLLM replica pair and its routing proxy were retired with this cutover:
+# one process, one model, both cards.
+#
+# Tunables come from frank-medium.env unless FRANK_MEDIUM_ENV_FILE overrides it.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_MEDIUM_ENV_FILE:-$SCRIPT_DIR/frank-medium.env}"
+if [[ -f "$ENV_FILE" ]]; then
+    set -a
+    # shellcheck disable=SC1090
+    source "$ENV_FILE"
+    set +a
+fi
+
+# --- defaults (used when the env file does not set them) ------------------
+: "${LLAMA_BIN:=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server}"
+: "${MODEL_DIR:=/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF}"
+: "${MODEL_FILE:=Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
+: "${DRAFT_MODEL_DIR:=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF}"
+: "${DRAFT_MODEL_FILE:=Qwen3.8-27B-DFlash2-Q8_0.gguf}"
+: "${ALIAS:=frank-medium}"
+: "${BIND_ADDR:=0.0.0.0}"
+: "${LISTEN_PORT:=7770}"
+
+# Devices.  HIP_VISIBLE_DEVICES hides the Raphael iGPU (HIP device 2) so nothing
+# can drift onto shared system RAM; DEVICES is what llama.cpp splits across.
+: "${HIP_VISIBLE_DEVICES:=0,1}"
+: "${DEVICES:=ROCm0,ROCm1}"
+: "${SPLIT_MODE:=layer}"
+: "${TENSOR_SPLIT:=1,1}"
+: "${NGL:=999}"
+: "${FIT:=off}"
+: "${FLASH_ATTN:=on}"
+
+: "${CTX_SIZE:=131072}"
+: "${PARALLEL:=1}"
+: "${BATCH_SIZE:=2048}"
+: "${UBATCH_SIZE:=512}"
+: "${NUM_THREADS:=8}"
+: "${THREAD_BATCH:=8}"
+: "${CACHE_TYPE_K:=f16}"
+: "${CACHE_TYPE_V:=f16}"
+: "${CTX_CHECKPOINTS:=8}"
+: "${CACHE_RAM:=8192}"
+
+# Sampling defaults for silent clients: the values the vLLM predecessor served.
+: "${TEMPERATURE:=0.7}"
+: "${TOP_P:=0.95}"
+: "${TOP_K:=20}"
+: "${MIN_P:=0.0}"
+: "${PRESENCE_PENALTY:=0.0}"
+
+: "${REASONING:=on}"
+: "${REASONING_FORMAT:=deepseek}"
+: "${REASONING_BUDGET:=-1}"
+: "${CHAT_TEMPLATE_FILE:=}"
+: "${CHAT_TEMPLATE_KWARGS:={\"enable_thinking\":true,\"preserve_thinking\":true}}"
+
+# Speculative decoding: draft-dflash or draft-dspark (both are MODEL drafters --
+# a file, not a flag), or draft-mtp (the target GGUF's own MTP head, no draft
+# file).  The drafter's own tuning knobs (n-min, p-min, p-split, poll) are left
+# at the build's defaults: they were only ever passed for dspark, and dspark is
+# not the deployed drafter.
+: "${SPEC_TYPE:=draft-dflash}"
+: "${SPEC_DRAFT_N_MAX:=7}"
+: "${SPEC_DRAFT_NGL:=999}"
+: "${SPEC_DRAFT_DEVICE:=}"
+: "${SPEC_DRAFT_TYPE_K:=f16}"
+: "${SPEC_DRAFT_TYPE_V:=f16}"
+: "${SPEC_NGRAM_N_MATCH:=24}"
+: "${SPEC_NGRAM_N_MIN:=48}"
+: "${SPEC_NGRAM_N_MAX:=64}"
+
+NO_MMPROJ="${NO_MMPROJ:-0}"
+: "${MM_PROJ_FILE:=mmproj-BF16.gguf}"
+
+export HIP_VISIBLE_DEVICES
+
+MODEL="$MODEL_DIR/$MODEL_FILE"
+DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
+
+if [[ ! -x "$LLAMA_BIN" ]]; then
+    echo "missing executable runtime: $LLAMA_BIN" >&2
+    echo "build it with: bash $SCRIPT_DIR/build-llama-rdna4.sh" >&2
+    exit 1
+fi
+if [[ ! -r "$MODEL" ]]; then
+    echo "missing or unreadable target model: $MODEL" >&2
+    echo "fetch it with: bash $SCRIPT_DIR/download-frank-medium-gguf.sh" >&2
+    exit 1
+fi
+if [[ "$NO_MMPROJ" != "1" ]]; then
+    MM_PROJ="$MODEL_DIR/$MM_PROJ_FILE"
+    if [[ ! -r "$MM_PROJ" ]]; then
+        echo "vision is enabled but the projector is missing: $MM_PROJ" >&2
+        echo "fetch it with: bash $SCRIPT_DIR/download-frank-medium-gguf.sh" >&2
+        exit 1
+    fi
+fi
+if [[ -n "$CHAT_TEMPLATE_FILE" && ! -r "$CHAT_TEMPLATE_FILE" ]]; then
+    echo "missing or unreadable chat template: $CHAT_TEMPLATE_FILE" >&2
+    exit 1
+fi
+
+# --- speculative decoding -------------------------------------------------
+SPEC_ARGS=()
+if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
+    SPEC_ARGS=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
+    if [[ "$SPEC_TYPE" == *dflash* || "$SPEC_TYPE" == *dspark* ]]; then
+        if [[ ! -r "$DRAFT_MODEL" ]]; then
+            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
+            exit 1
+        fi
+        SPEC_ARGS+=(
+            --spec-draft-model "$DRAFT_MODEL"
+            --spec-draft-ngl "$SPEC_DRAFT_NGL"
+            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
+            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
+        )
+    fi
+    # ngram-mod is draftless: it needs no draft context and its window is its
+    # own (--spec-ngram-mod-*), so it can ride in the chain beside a model
+    # drafter. It also costs no VRAM -- a CPU hash table.
+    if [[ "$SPEC_TYPE" == *ngram* ]]; then
+        SPEC_ARGS+=(
+            --spec-ngram-mod-n-match "$SPEC_NGRAM_N_MATCH"
+            --spec-ngram-mod-n-min "$SPEC_NGRAM_N_MIN"
+            --spec-ngram-mod-n-max "$SPEC_NGRAM_N_MAX"
+        )
+    fi
+    if [[ -n "$SPEC_DRAFT_DEVICE" ]]; then
+        SPEC_ARGS+=(--spec-draft-device "$SPEC_DRAFT_DEVICE")
+    fi
+fi
+
+# --- template + placement -------------------------------------------------
+TEMPLATE_ARGS=()
+if [[ -n "$CHAT_TEMPLATE_FILE" ]]; then
+    TEMPLATE_ARGS+=(--chat-template-file "$CHAT_TEMPLATE_FILE")
+fi
+TEMPLATE_ARGS+=(--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS")
+
+PLACEMENT_ARGS=(--device "$DEVICES" --split-mode "$SPLIT_MODE" --tensor-split "$TENSOR_SPLIT")
+
+MMPROJ_ARGS=()
+if [[ "$NO_MMPROJ" == "1" ]]; then
+    MMPROJ_ARGS=(--no-mmproj)
+else
+    # No autoload in this build: a server without the flag starts text-only
+    # and answers image requests with a 500, so name the projector.
+    MMPROJ_ARGS=(--mmproj "$MM_PROJ")
+fi
+exec "$LLAMA_BIN" \
+    --model "$MODEL" \
+    --alias "$ALIAS" \
+    --host "$BIND_ADDR" \
+    --port "$LISTEN_PORT" \
+    --jinja \
+    --gpu-layers "$NGL" \
+    --fit "$FIT" \
+    --parallel "$PARALLEL" \
+    --ctx-size "$CTX_SIZE" \
+    --batch-size "$BATCH_SIZE" \
+    --ubatch-size "$UBATCH_SIZE" \
+    --flash-attn "$FLASH_ATTN" \
+    --cache-type-k "$CACHE_TYPE_K" \
+    --cache-type-v "$CACHE_TYPE_V" \
+    --threads "$NUM_THREADS" \
+    --threads-batch "$THREAD_BATCH" \
+    --ctx-checkpoints "$CTX_CHECKPOINTS" \
+    --cache-ram "$CACHE_RAM" \
+    --cache-prompt \
+    --metrics \
+    --temp "$TEMPERATURE" \
+    --top-p "$TOP_P" \
+    --top-k "$TOP_K" \
+    --min-p "$MIN_P" \
+    --presence-penalty "$PRESENCE_PENALTY" \
+    --reasoning "$REASONING" \
+    --reasoning-format "$REASONING_FORMAT" \
+    --reasoning-budget "$REASONING_BUDGET" \
+    "${TEMPLATE_ARGS[@]}" \
+    "${PLACEMENT_ARGS[@]}" \
+    "${MMPROJ_ARGS[@]}" \
+    "${SPEC_ARGS[@]}"
@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""
+Decode throughput benchmark for frank-dragoon.
+Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
+generates 256 tokens, measures generation tok/s.
+
+Each --threads-batch value requires a cold server load (~3 min).
+"""
+import json, time, urllib.request, urllib.error, subprocess, os, sys
+
+MODEL = "/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"
+SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
+PORT = 7770
+N_GEN = 256
+PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
+ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
+SERVER_DIR = os.path.dirname(SERVE)
+ROCM_PATH = "/opt/rocm"
+
+def make_env():
+    """Build the same env as serve-frank-dragoon.sh."""
+    env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
+    
+    # Source env file for tunables
+    if os.path.exists(ENV_FILE):
+        with open(ENV_FILE) as f:
+            for line in f:
+                line = line.strip()
+                if line and not line.startswith("#") and "=" in line:
+                    k, v = line.split("=", 1)
+                    env[k] = v
+    
+    # Build LD_LIBRARY_PATH
+    lib_path = f"{SERVER_DIR}:{ROCM_PATH}/lib:{ROCM_PATH}/lib/rocm_sysdeps/lib:{ROCM_PATH}/lib/llvm/lib"
+    if os.environ.get("LD_LIBRARY_PATH"):
+        lib_path += f":{os.environ['LD_LIBRARY_PATH']}"
+    env["LD_LIBRARY_PATH"] = lib_path
+    env["ROCM_PATH"] = ROCM_PATH
+    
+    return env
+
+ENV = make_env()
+
+def kill_all():
+    subprocess.run(["pkill", "-9", "llama-server"], capture_output=True)
+    time.sleep(3)
+
+def wait_ready():
+    for i in range(600):
+        try:
+            r = urllib.request.urlopen(f"http://127.0.0.1:{PORT}/health", timeout=2)
+            if json.loads(r.read()).get("status") == "ok":
+                return True
+        except:
+            pass
+        time.sleep(1)
+    return False
+
+def send_request(prompt, max_tokens=N_GEN):
+    """Send chat completion, measure generation throughput."""
+    body = json.dumps({
+        "model": "frank-dragoon",
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": max_tokens,
+        "stream": False,
+        "temperature": 0.7,
+        "top_p": 0.8
+    }).encode()
+    
+    t0 = time.time()
+    req = urllib.request.Request(
+        f"http://127.0.0.1:{PORT}/v1/chat/completions",
+        data=body,
+        headers={"Content-Type": "application/json"},
+        method="POST"
+    )
+    r = urllib.request.urlopen(req, timeout=1200)
+    resp = json.loads(r.read())
+    wall = time.time() - t0
+    
+    usage = resp.get("usage", {})
+    pt = usage.get("prompt_tokens", 0)
+    gt = usage.get("completion_tokens", 0)
+    gtps = gt / wall if wall > 0 else 0
+    return pt, gt, wall, gtps
+
+def load_and_extend_prompt():
+    """Load prompt and extend to ~8K tokens if needed."""
+    with open(PROMPT_FILE) as f:
+        text = f.read()
+    
+    # Count approximate tokens (rough: ~4 chars per token for code)
+    approx_tokens = len(text) / 4.0
+    needed = 8200
+    
+    if approx_tokens < needed:
+        repeats = needed // max(int(approx_tokens), 1) + 1
+        text = (text + "\n") * repeats
+        words = text.split()[:needed]
+        text = " ".join(words)
+    
+    return text
+
+def main():
+    tb_values = [int(x) for x in sys.argv[1:]] if len(sys.argv) > 1 else [16, 32, 64, 128]
+    results = []
+    
+    prompt = load_and_extend_prompt()
+    n_words = len(prompt.split())
+    print(f"Prompt: {n_words} words (~{n_words} tokens), {len(prompt)} chars\n", flush=True)
+    
+    for tb in tb_values:
+        print(f"\n{'='*60}")
+        print(f"threads-batch = {tb}")
+        print(f"{'='*60}", flush=True)
+        
+        kill_all()
+        print("  Starting server...", flush=True)
+        
+        lib_path = ENV.get("LD_LIBRARY_PATH", "")
+        cmd = [
+            "/usr/bin/env",
+            "HSA_OVERRIDE_GFX_VERSION=11.5.1",
+            "HIP_VISIBLE_DEVICES=0",
+            f"LD_LIBRARY_PATH={lib_path}",
+            f"ROCM_PATH={ROCM_PATH}",
+            SERVE, "--model", MODEL, "--port", str(PORT), "--jinja",
+            "--device", "ROCm0", "--gpu-layers", "999", "--fit", "off",
+            "--threads", "16", "--threads-batch", str(tb),
+            "--parallel", "2", "--ctx-size", "131072",
+            "--batch-size", "4096", "--ubatch-size", "4096",
+            "--flash-attn", "on",
+            "--cache-type-k", "f16", "--cache-type-v", "f16",
+            "--cache-prompt", "--cache-idle-slots", "--metrics"
+        ]
+        
+        srv = subprocess.Popen(cmd, env=ENV, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
+        
+        ready = wait_ready()
+        if not ready:
+            print("  FAILED: server did not become ready", flush=True)
+            srv.terminate()
+            srv.wait()
+            for line in (srv.stdout.readlines() or [])[-20:]:
+                print(f"  LOG: {line.rstrip()}", flush=True)
+            continue
+        
+        print("  Server ready.", flush=True)
+        
+        # Warmup (small request)
+        try:
+            send_request("Hi", 16)
+        except Exception as e:
+            print(f"  Warmup failed: {e}", flush=True)
+            srv.terminate()
+            srv.wait()
+            continue
+        
+        # Benchmark
+        print(f"  Generating {N_GEN} tokens...", flush=True)
+        try:
+            pt, gt, wall, gtps = send_request(prompt, N_GEN)
+            print(f"  {gt}/{N_GEN} gen tokens in {wall:.1f}s = {gtps:.1f} tok/s")
+            print(f"  Prompt tokens: {pt}")
+            results.append({"tb": tb, "wall": wall, "gtps": gtps, "gt": gt, "pt": pt})
+        except Exception as e:
+            print(f"  FAILED: {e}", flush=True)
+        
+        srv.terminate()
+        srv.wait()
+        time.sleep(2)
+    
+    # Summary
+    print(f"\n{'='*60}")
+    print("SUMMARY")
+    print(f"{'='*60}")
+    for r in results:
+        print(f"  tb={r['tb']:>3}: {r['gt']} gen tokens, {r['wall']:.1f}s, {r['gtps']:.1f} tok/s")
+    
+    if results:
+        best = max(results, key=lambda r: r['gtps'])
+        print(f"\nBest: tb={best['tb']} at {best['gtps']:.1f} tok/s")
+    
+    with open("/tmp/bench_decode_results.txt", "w") as f:
+        f.write("tb,gen_tokens,wall_s,gen_tps,prompt_tokens\n")
+        for r in results:
+            f.write(f"{r['tb']},{r['gt']},{r['wall']:.1f},{r['gtps']:.1f},{r['pt']}\n")
+    
+    print(f"\nResults: /tmp/bench_decode_results.txt")
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,268 @@
+#!/usr/bin/env python3
+"""
+Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
+
+Tests prompt-processing throughput with 16K token prompts (manageable request size).
+Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
+
+Usage:
+    # Generate prompts: python3 scripts/bench-ngram-mod.py generate /tmp/prompts/
+    # Run: python3 scripts/bench-ngram-mod.py /tmp/prompts/
+"""
+
+import json
+import os
+import sys
+import time
+import subprocess
+import urllib.request
+import urllib.error
+from pathlib import Path
+
+MODEL = "/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"
+CHAT_TEMPLATE = "/home/dank/serverAppConfigs/scripts/chat-template.jinja"
+BUILD_DIR = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
+LLAMA_BIN = os.path.join(BUILD_DIR, "llama-server")
+
+# 16K tokens ~= 64KB text
+TARGET_PROMPT_TOKENS = 16000
+TOKEN_CHAR_RATIO = 4
+TARGET_PROMPT_BYTES = TARGET_PROMPT_TOKENS * TOKEN_CHAR_RATIO
+
+CONTEXT = 65536
+BATCH = 4096
+UBATCH = 4096
+THREADS = 16
+THREAD_BATCH = 32
+NGRAM_N_MAX = 24
+NGRAM_N_MIN = 4
+TEST_TOKENS = 128
+SERVER_WAIT = 300
+
+
+def start_server(port, spec_type):
+    cmd = [
+        LLAMA_BIN,
+        "--model", MODEL,
+        "--chat-template-file", CHAT_TEMPLATE,
+        "--jinja",
+        "--device", "ROCm0",
+        "--gpu-layers", "999",
+        "--parallel", "1",
+        "--ctx-size", str(CONTEXT),
+        "--batch-size", str(BATCH),
+        "--ubatch-size", str(UBATCH),
+        "--fit", "off",
+        "--flash-attn", "on",
+        "--cache-type-k", "f16",
+        "--cache-type-v", "f16",
+        "--threads", str(THREADS),
+        "--threads-batch", str(THREAD_BATCH),
+        "--cache-prompt",
+        "--cache-idle-slots",
+        "--metrics",
+        "--host", "127.0.0.1",
+        "--port", str(port),
+        "--temp", "0",
+    ]
+    if spec_type == "ngram-mod":
+        cmd.extend([
+            "--spec-type", "ngram-mod",
+            "--spec-ngram-mod-n-max", str(NGRAM_N_MAX),
+            "--spec-ngram-mod-n-min", str(NGRAM_N_MIN),
+        ])
+    else:
+        cmd.extend(["--spec-type", "none"])
+
+    env = os.environ.copy()
+    env["ROCM_PATH"] = "/opt/rocm"
+    env["LD_LIBRARY_PATH"] = f"{BUILD_DIR}:/opt/rocm/lib:/opt/rocm/lib/rocm_sysdeps/lib:/opt/rocm/lib/llvm/lib"
+    env["HSA_OVERRIDE_GFX_VERSION"] = "11.5.1"
+    env["HIP_VISIBLE_DEVICES"] = "0"
+
+    proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+    print(f"  Waiting for server on port {port} (spec={spec_type})...")
+    for i in range(SERVER_WAIT):
+        try:
+            req = urllib.request.urlopen(f"http://127.0.0.1:{port}/health", timeout=2)
+            resp = json.loads(req.read().decode())
+            if resp.get("status") == "ok":
+                print(f"  Server ready after {i+1}s")
+                return proc
+        except:
+            time.sleep(1)
+
+    proc.terminate()
+    out, err = proc.communicate()
+    print(f"ERROR: Server failed to start in {SERVER_WAIT}s")
+    print(err.decode()[-2000:])
+    sys.exit(1)
+
+
+def run_test(prompt_text, spec_type, port):
+    """Send a request and extract prompt processing metrics."""
+    data = json.dumps({
+        "model": "frank-dragoon",
+        "messages": [{"role": "user", "content": prompt_text}],
+        "max_tokens": TEST_TOKENS,
+        "stream": False,
+        "temperature": 0,
+    }).encode()
+
+    req = urllib.request.Request(
+        f"http://127.0.0.1:{port}/v1/chat/completions",
+        data=data,
+        headers={"Content-Type": "application/json"},
+    )
+
+    start = time.monotonic()
+    resp = urllib.request.urlopen(req, timeout=600)
+    elapsed = time.monotonic() - start
+
+    result = json.loads(resp.read().decode())
+    timings = result.get("timings", {})
+    usage = result.get("usage", {})
+
+    return {
+        "prompt_tokens": usage.get("prompt_tokens", 0),
+        "prompt_ms": timings.get("prompt_ms", 0),
+        "prompt_per_second": timings.get("prompt_per_second", 0),
+        "prompt_per_token_ms": timings.get("prompt_per_token_ms", 0),
+        "predicted_per_second": timings.get("predicted_per_second", 0),
+        "total_time_s": elapsed,
+        "cache_n": timings.get("cache_n", 0),
+    }
+
+
+def main():
+    if len(sys.argv) < 2:
+        print(f"Usage: {sys.argv[0]} <prompt_dir>")
+        sys.exit(1)
+
+    prompt_dir = Path(sys.argv[1])
+    prompt_dir.mkdir(parents=True, exist_ok=True)
+
+    code_prompt_file = prompt_dir / "prompt_code.txt"
+    prose_prompt_file = prompt_dir / "prompt_prose.txt"
+
+    # Generate prompts if they don't exist
+    if not code_prompt_file.exists() or not prose_prompt_file.exists():
+        print("Generating prompts...")
+        # Generate deterministic prompts
+        # Code prompt: structured algorithms + data generation + config schemas
+        code_lines = []
+        for i in range(50):
+            code_lines.append(f"# Algorithm {i}\n")
+            code_lines.append(f"def algo_{i}(x, y):\n")
+            code_lines.append(f"    \"\"\"Algorithm {i} processing.\"\"\"\n")
+            code_lines.append(f"    result = 0\n")
+            code_lines.append(f"    for j in range({i * 10}):\n")
+            code_lines.append(f"        result += x * y + j\n")
+            code_lines.append(f"    return result\n\n")
+        code_text = "".join(code_lines)
+        # Pad/truncate to target
+        while len(code_text) < TARGET_PROMPT_BYTES:
+            code_text += "# Additional algorithm block\n"
+            code_text += f"def extra_{len(code_text)//100}():\n"
+            code_text += "    x = 1\n"
+            code_text += "    y = 2\n"
+            code_text += "    return x + y\n\n"
+        code_text = code_text[:TARGET_PROMPT_BYTES]
+
+        # Prose prompt: historical/scientific/philosophical text
+        prose_text = ""
+        topics = [
+            "The Renaissance brought profound transformation to European civilization. Artists and scholars pushed the boundaries of human expression.",
+            "Quantum mechanics describes the behavior of matter and energy at atomic scales. The uncertainty principle limits measurement precision.",
+            "Ethical philosophy grapples with questions of right and wrong. Utilitarianism judges actions by their consequences and the happiness they produce.",
+            "Mountains rise like ancient sentinels, their peaks lost in clouds that swirl and dance across the sky above the valleys below.",
+            "The ocean whispers secrets to the shore, each wave carrying fragments of distant lands across the water before retreating back to sea.",
+        ]
+        for _ in range(500):
+            prose_text += f"{topics[len(prose_text) % len(topics)]} "
+        prose_text = prose_text[:TARGET_PROMPT_BYTES]
+
+        code_prompt_file.write_text(code_text)
+        prose_prompt_file.write_text(prose_text)
+        print(f"  Code: {code_prompt_file.stat().st_size:,} bytes")
+        print(f"  Prose: {prose_prompt_file.stat().st_size:,} bytes")
+
+    print("=" * 60)
+    print("  N-gram Mod Benchmark — frank-dragoon")
+    print(f"  STRIX ROCmFP4 · 64K context · 16K prompts")
+    print("=" * 60)
+
+    results = []
+    port_base = 7772
+    port_ngram = 7773
+
+    tests = [
+        ("code", "none", port_base, code_prompt_file),
+        ("prose", "none", port_ngram, prose_prompt_file),
+        ("code", "ngram-mod", port_base, code_prompt_file),
+        ("prose", "ngram-mod", port_ngram, prose_prompt_file),
+    ]
+
+    for prompt_type, spec_type, port, pfile in tests:
+        print(f"\n=== {spec_type} ({prompt_type}) ===")
+        print(f"  File: {pfile} ({pfile.stat().st_size:,} bytes)")
+        print(f"  Spec: {spec_type}")
+        if spec_type == "ngram-mod":
+            print(f"  n_max={NGRAM_N_MAX}, n_min={NGRAM_N_MIN}")
+
+        proc = start_server(port, spec_type)
+
+        try:
+            # Warm-up
+            print("  Warm-up...")
+            run_test("Hello test", spec_type, port)
+
+            # Full prompt
+            print(f"  Sending {TARGET_PROMPT_TOKENS} token prompt...")
+            with open(pfile) as f:
+                prompt_text = f.read()
+            result = run_test(prompt_text, spec_type, port)
+
+            tps = result["prompt_per_second"]
+            print(f"  Tokens:  {result['prompt_tokens']}")
+            print(f"  Prompt:  {result['prompt_ms']:.0f}ms")
+            print(f"  TPS:     {tps:.1f} tok/s")
+            print(f"  Total:   {result['total_time_s']:.1f}s")
+
+            results.append({
+                "prompt_type": prompt_type,
+                "spec_type": spec_type,
+                "prompt_tokens": result["prompt_tokens"],
+                "prompt_ms": result["prompt_ms"],
+                "tps": tps,
+                "total_time_s": result["total_time_s"],
+            })
+
+        finally:
+            proc.terminate()
+            try:
+                proc.wait(timeout=5)
+            except:
+                proc.kill()
+
+    # Summary
+    print("\n" + "=" * 60)
+    print("  RESULTS SUMMARY")
+    print("=" * 60)
+
+    for ptype in ["code", "prose"]:
+        baseline = next((r for r in results if r["prompt_type"] == ptype and r["spec_type"] == "none"), None)
+        ngram = next((r for r in results if r["prompt_type"] == ptype and r["spec_type"] == "ngram-mod"), None)
+        if baseline and ngram:
+            gain = ((ngram["tps"] - baseline["tps"]) / baseline["tps"]) * 100 if baseline["tps"] > 0 else 0
+            print(f"\n{ptype.upper()} PROMPT:")
+            print(f"  Baseline TPS: {baseline['tps']:.1f} tok/s ({baseline['prompt_ms']:.0f}ms)")
+            print(f"  N-gram TPS:   {ngram['tps']:.1f} tok/s ({ngram['prompt_ms']:.0f}ms)")
+            print(f"  Speedup:      {gain:+.1f}%")
+
+    print("\n" + "=" * 60)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,304 @@
+#!/usr/bin/env bash
+# Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
+#
+# This tests prompt-processing throughput at 64k context with two distinct
+# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
+# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
+#
+# Run on Dankclaw after stopping the production service:
+#   sudo systemctl stop frank-dragoon
+#   ./scripts/bench-ngram-mod.sh
+#   sudo systemctl start frank-dragoon
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
+if [[ -f "$ENV_FILE" ]]; then
+    set -a; source "$ENV_FILE"; set +a
+fi
+
+: "${LLAMA_BIN:=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server}"
+: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
+: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
+: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
+: "${CHAT_TEMPLATE_KWARGS:={}}"
+: "${BATCH_SIZE:=4096}"
+: "${UBATCH_SIZE:=4096}"
+: "${CTX_SIZE:=65536}"
+: "${NUM_THREADS:=16}"
+: "${THREAD_BATCH:=32}"
+: "${CACHE_TYPE_K:=f16}"
+: "${CACHE_TYPE_V:=f16}"
+: "${FIT:=off}"
+: "${TEMPERATURE:=0.7}"
+: "${TOP_P:=0.8}"
+: "${TOP_K:=20}"
+: "${MIN_P:=0.0}"
+: "${REASONING:=on}"
+: "${REASONING_FORMAT:=deepseek}"
+
+MODEL="$MODEL_DIR/$MODEL_FILE"
+HOSTNAME=$(hostname 2>/dev/null || echo "dankclaw")
+
+# Prompt files (generated by generate-prompts.py or passed via args)
+PROMPT_CODE="${1:-}"
+PROMPT_PROSE="${2:-}"
+
+if [[ -z "$PROMPT_CODE" || -z "$PROMPT_PROSE" ]]; then
+    echo "ERROR: Need two prompt file paths."
+    echo "Usage: $0 <prompt_code.txt> <prompt_prose.txt>"
+    echo "Generate with: python3 scripts/generate-prompts.py"
+    exit 1
+fi
+
+if [[ ! -r "$MODEL" ]]; then
+    echo "missing or unreadable model: $MODEL" >&2
+    exit 1
+fi
+
+SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
+ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
+LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+
+# Scratch ports
+PORT_NGRAM=7771
+PORT_BASELINE=7772
+
+cleanup() {
+    echo "Cleaning up servers..."
+    # Kill any leftover llama-server on our ports
+    for port in $PORT_NGRAM $PORT_BASELINE; do
+        fuser -k "$port"/tcp 2>/dev/null || true
+    done
+}
+trap cleanup EXIT
+
+wait_for_server() {
+    local port=$1
+    local max_wait=120
+    local elapsed=0
+    while (( elapsed < max_wait )); do
+        if curl -s "http://127.0.0.1:$port/health" | grep -q '"running"'; then
+            return 0
+        fi
+        sleep 1
+        (( elapsed++ ))
+    done
+    echo "ERROR: Server on port $port did not start within ${max_wait}s" >&2
+    return 1
+}
+
+get_prompt_tokens() {
+    local file=$1
+    # Count tokens approximately (1 token ~ 4 chars for English/code)
+    wc -c < "$file" | awk '{printf "%d", $1/4}'
+}
+
+run_single_test() {
+    local port=$1
+    local spec_type="$2"
+    local prompt_file=$3
+    local label=$4
+    local ngram_n_max="${5:-24}"
+    local ngram_n_min="${6:-4}"
+    
+    echo "=== $label ==="
+    echo "  spec-type: $spec_type"
+    echo "  ngram-mod n-max: $ngram_n_max (if enabled)"
+    echo "  ngram-mod n-min: $ngram_n_min (if enabled)"
+    echo "  prompt: $prompt_file ($(wc -c < "$prompt_file") bytes)"
+    
+    # Build command
+    local cmd=(
+        /usr/bin/env
+        ROCM_PATH="$ROCM_PATH"
+        HIP_VISIBLE_DEVICES=0
+        LD_LIBRARY_PATH="$LIBRARY_PATH"
+        HSA_OVERRIDE_GFX_VERSION="${HSA_OVERRIDE_GFX_VERSION:-11.5.1}"
+        "$LLAMA_BIN"
+        --model "$MODEL"
+        --chat-template-file "$CHAT_TEMPLATE_FILE"
+        --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS"
+        --host 127.0.0.1
+        --port "$port"
+        --jinja
+        --device ROCm0
+        --gpu-layers 999
+        --parallel 1
+        --ctx-size "$CTX_SIZE"
+        --batch-size "$BATCH_SIZE"
+        --ubatch-size "$UBATCH_SIZE"
+        --fit "$FIT"
+        --flash-attn on
+        --cache-type-k "$CACHE_TYPE_K"
+        --cache-type-v "$CACHE_TYPE_V"
+        --threads "$NUM_THREADS"
+        --threads-batch "$THREAD_BATCH"
+        --cache-prompt
+        --cache-idle-slots
+        --metrics
+        --temp "$TEMPERATURE"
+        --top-p "$TOP_P"
+        --top-k "$TOP_K"
+        --min-p "$MIN_P"
+        --reasoning "$REASONING"
+        --reasoning-format "$REASONING_FORMAT"
+        --reasoning-budget -1
+    )
+    
+    # Add spec flags
+    if [[ "$spec_type" == "ngram-mod" ]]; then
+        cmd+=(
+            --spec-type "ngram-mod"
+            --spec-ngram-mod-n-max "$ngram_n_max"
+            --spec-ngram-mod-n-min "$ngram_n_min"
+        )
+    else
+        cmd+=(
+            --spec-type "none"
+        )
+    fi
+    
+    # Start server with stderr to log
+    local log_file="/tmp/ngram-bench-${label//[^a-zA-Z0-9]/-}.log"
+    echo "  Starting server (log: $log_file)..."
+    
+    "${cmd[@]}" 2> "$log_file" &
+    local server_pid=$!
+    
+    if ! wait_for_server "$port"; then
+        echo "ERROR: Server failed to start" >&2
+        kill "$server_pid" 2>/dev/null || true
+        return 1
+    fi
+    
+    echo "  Server ready."
+    
+    # Send prompt via API
+    echo "  Sending prompt..."
+    local prompt_text
+    prompt_text=$(cat "$prompt_file")
+    
+    local response
+    response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
+        -H "Content-Type: application/json" \
+        -d "{
+            \"model\": \"frank-dragoon\",
+            \"prompt\": \"$prompt_text\",
+            \"max_tokens\": 128,
+            \"temperature\": 0,
+            \"stream\": false
+        }")
+    
+    # Extract metrics
+    local prompt_tokens prompt_eval_duration_ms total_duration_ms
+    prompt_tokens=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['prompt_eval_count'])" 2>/dev/null || echo "0")
+    prompt_eval_duration_ms=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['prompt_eval_duration_ms'])" 2>/dev/null || echo "0")
+    total_duration_ms=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['total_duration_ms'])" 2>/dev/null || echo "0")
+    
+    # Calculate throughput
+    local prompt_tps=0
+    if [[ "$prompt_eval_duration_ms" -gt 0 ]]; then
+        prompt_tps=$(python3 -c "print(f'{($prompt_tokens / $prompt_eval_duration_ms) * 1000:.1f}')")
+    fi
+    
+    echo "  Prompt tokens:  $prompt_tokens"
+    echo "  Prompt eval:    ${prompt_eval_duration_ms}ms"
+    echo "  Prompt TPS:     $prompt_tps tok/s"
+    echo "  Total duration: ${total_duration_ms}ms"
+    
+    # Kill server
+    kill "$server_pid" 2>/dev/null || true
+    sleep 2
+    
+    echo "  Done."
+    echo "---"
+    
+    # Output structured results for parsing
+    echo "RESULT|$label|$prompt_file|$prompt_tokens|$prompt_eval_duration_ms|$total_duration_ms|$prompt_tps"
+}
+
+echo "╔══════════════════════════════════════════════════════════╗"
+echo "║  N-gram Mod Benchmark — frank-dragoon             ║"
+echo "║  STRIX ROCmFP4 · 64k context · two prompts             ║"
+echo "╚══════════════════════════════════════════════════════════╝"
+echo ""
+echo "Host: $HOSTNAME"
+echo "Model: $MODEL"
+echo "Build: $(basename "$LLAMA_BIN")"
+echo "Context: $CTX_SIZE"
+echo ""
+
+# Run baseline (no spec)
+echo "=== BASELINE (no speculative decoding) ==="
+echo ""
+
+BASELINE_CODE_RESULT=$(run_single_test $PORT_BASELINE "none" "$PROMPT_CODE" "Baseline-Code" 24 4)
+BASELINE_PROSE_RESULT=$(run_single_test $PORT_NGRAM "none" "$PROMPT_PROSE" "Baseline-Prose" 24 4)
+
+echo ""
+echo "=== NGRAM-MOD (speculative decoding enabled) ==="
+echo ""
+
+# Run n-gram mod
+NGRAM_CODE_RESULT=$(run_single_test $PORT_NGRAM "ngram-mod" "$PROMPT_CODE" "Ngram-Code" 24 4)
+NGRAM_PROSE_RESULT=$(run_single_test $PORT_BASELINE "ngram-mod" "$PROMPT_PROSE" "Ngram-Prose" 24 4)
+
+echo ""
+echo "╔══════════════════════════════════════════════════════════╗"
+echo "║  RESULTS SUMMARY                                       ║"
+echo "╚══════════════════════════════════════════════════════════╝"
+echo ""
+
+# Parse and display results
+python3 << 'PYEOF'
+import sys
+
+results = []
+for line in sys.stdin:
+    line = line.strip()
+    if not line.startswith("RESULT|"):
+        continue
+    parts = line.split("|")
+    if len(parts) != 8:
+        continue
+    label, prompt_file, prompt_tokens, prompt_eval_ms, total_ms, tps = parts[1], parts[2], int(parts[3]), int(parts[4]), int(parts[5]), float(parts[6])
+    results.append({
+        'label': label,
+        'prompt_file': prompt_file,
+        'prompt_tokens': prompt_tokens,
+        'prompt_eval_ms': prompt_eval_ms,
+        'total_ms': total_ms,
+        'tps': tps
+    })
+
+# Compare baseline vs ngram for each prompt type
+for code_result in results:
+    if not code_result['label'].startswith('Ngram-'):
+        continue
+    prompt_type = code_result['label'].replace('Ngram-', '').replace('Ngram', 'Baseline').replace('-', '-')
+    baseline = next((r for r in results if r['label'] == 'Baseline-' + prompt_type), None)
+    if not baseline:
+        continue
+    
+    spec = code_result['label'] == 'Ngram-' + prompt_type
+    
+    print(f"Prompt: {prompt_type}")
+    print(f"  Tokens:   {code_result['prompt_tokens']}")
+    if spec:
+        print(f"  Spec:     ngram-mod (n_max=24, n_min=4)")
+        print(f"  Prompt:   {code_result['prompt_eval_ms']}ms  |  {code_result['tps']:>10.1f} tok/s")
+        baseline_tps = baseline['tps']
+        if baseline_tps > 0:
+            improvement = ((code_result['tps'] - baseline_tps) / baseline_tps) * 100
+            print(f"  Speedup:  {improvement:+.1f}% vs baseline ({baseline_tps:.1f} tok/s)")
+    else:
+        print(f"  Spec:     none (baseline)")
+        print(f"  Prompt:   {baseline['prompt_eval_ms']}ms  |  {baseline['tps']:>10.1f} tok/s")
+    print()
+PYEOF
+
+echo ""
+echo "Raw data lines above (RESULT|... format) can be parsed for automation."
+echo ""
@@ -0,0 +1,229 @@
+#!/usr/bin/env python3
+"""Benchmark decode throughput across speculative decoding configs.
+
+Sweeps 4 configs: none, ngram-mod, draft-mtp, draft-mtp+ngram-mod.
+Each config generates 16k tokens from a minimal prompt at temp 0.
+
+Runs ON the target host (Dankclaw). Service management uses local sudo.
+Usage:
+    python3 bench-spec-decode.py [--max-tokens N] [--host H] [--port P]
+"""
+import json
+import subprocess
+import sys
+import time
+import urllib.request
+import urllib.error
+
+SERVICE = "frank-dragoon"
+ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
+ALIAS = "frank-dragoon"
+DEFAULT_HOST = "localhost"
+DEFAULT_PORT = 7770
+DEFAULT_MAX_TOKENS = 16384
+HEALTH_TIMEOUT = 1200  # 20 min for cold load
+HEALTH_INTERVAL = 5
+
+PROMPT = (
+    "Write a comprehensive, detailed essay about the complete history of "
+    "computing, from the earliest mechanical calculators through modern AI. "
+    "Cover every major era in detail, including key figures, machines, and "
+    "breakthroughs. Be as thorough as possible."
+)
+
+CONFIGS = [
+    ("none", "none"),
+    ("ngram-mod", "ngram-mod"),
+    ("draft-mtp", "draft-mtp"),
+    ("draft-mtp+ngram-mod", "draft-mtp,ngram-mod"),
+]
+
+
+def local(cmd):
+    """Run a command locally (the script runs on the target host)."""
+    r = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=120)
+    if r.returncode != 0:
+        print(f"  WARNING: {cmd} -> exit {r.returncode}: {r.stderr.strip()}",
+              file=sys.stderr)
+    return r
+
+
+def sed_spec_type(value):
+    local(f"sudo sed -i 's/^SPEC_TYPE=.*/SPEC_TYPE={value}/' {ENV_FILE}")
+
+
+def wait_health(host, port, timeout=HEALTH_TIMEOUT):
+    start = time.time()
+    while time.time() - start < timeout:
+        try:
+            resp = urllib.request.urlopen(
+                f"http://{host}:{port}/health", timeout=10
+            )
+            if resp.status == 200:
+                return True
+        except (urllib.error.URLError, urllib.error.HTTPError, ConnectionError,
+                OSError):
+            pass
+        time.sleep(HEALTH_INTERVAL)
+    return False
+
+
+def benchmark(host, port, alias, max_tokens):
+    body = json.dumps({
+        "model": alias,
+        "messages": [{"role": "user", "content": PROMPT}],
+        "max_tokens": max_tokens,
+        "temperature": 0,
+        "stream": False,
+    }).encode()
+    req = urllib.request.Request(
+        f"http://{host}:{port}/v1/chat/completions",
+        data=body,
+        headers={"Content-Type": "application/json"},
+    )
+    t0 = time.time()
+    resp = urllib.request.urlopen(req, timeout=1200)
+    wall = time.time() - t0
+    data = json.loads(resp.read())
+    return data, wall
+
+
+def main():
+    import argparse
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--host", default=DEFAULT_HOST)
+    ap.add_argument("--port", type=int, default=DEFAULT_PORT)
+    ap.add_argument("--max-tokens", type=int, default=DEFAULT_MAX_TOKENS)
+    args = ap.parse_args()
+
+    # Read original SPEC_TYPE from the env file
+    r = local(f"grep '^SPEC_TYPE=' {ENV_FILE}")
+    original_spec = r.stdout.strip()
+    orig_val = (original_spec.split("=", 1)[1]
+                if "=" in original_spec else "draft-mtp,ngram-mod")
+    print(f"Original: {original_spec}")
+    print(f"Max tokens: {args.max_tokens}")
+    print(f"Configs: {[c[0] for c in CONFIGS]}")
+    print()
+
+    # Stop + mask the production service
+    print("Stopping + masking production service...")
+    local(f"sudo systemctl stop {SERVICE}")
+    local(f"sudo systemctl mask {SERVICE}")
+    time.sleep(2)
+
+    results = []
+    for name, spec_type in CONFIGS:
+        print(f"\n{'='*60}")
+        print(f"Config: {name} (SPEC_TYPE={spec_type})")
+        print(f"{'='*60}")
+
+        # Set SPEC_TYPE
+        sed_spec_type(spec_type)
+        print(f"Set SPEC_TYPE={spec_type}")
+
+        # Start the service
+        local(f"sudo systemctl unmask {SERVICE}")
+        local(f"sudo systemctl start {SERVICE}")
+        print("Starting service...")
+
+        # Wait for health
+        print(f"Waiting for health (up to {HEALTH_TIMEOUT}s)...")
+        if not wait_health(args.host, args.port):
+            print(f"FAILED: service did not become healthy for {name}")
+            results.append({
+                "config": name, "spec_type": spec_type,
+                "error": "health timeout",
+            })
+            local(f"sudo systemctl stop {SERVICE}")
+            local(f"sudo systemctl mask {SERVICE}")
+            continue
+
+        print("Healthy. Running benchmark...")
+
+        # Run benchmark
+        try:
+            data, wall = benchmark(
+                args.host, args.port, ALIAS, args.max_tokens
+            )
+        except Exception as e:
+            print(f"FAILED: {e}")
+            results.append({
+                "config": name, "spec_type": spec_type,
+                "error": str(e),
+            })
+            local(f"sudo systemctl stop {SERVICE}")
+            local(f"sudo systemctl mask {SERVICE}")
+            continue
+
+        timings = data.get("timings", {})
+        usage = data.get("usage", {})
+        pred_n = timings.get("predicted_n", usage.get("completion_tokens", 0))
+        pred_ms = timings.get("predicted_ms", 0)
+        pred_ps = timings.get("predicted_per_second", 0)
+        prompt_n = timings.get("prompt_n", usage.get("prompt_tokens", 0))
+        prompt_ms = timings.get("prompt_ms", 0)
+
+        print(f"  Generated:   {pred_n} tokens")
+        print(f"  Wall time:   {wall:.1f}s")
+        print(f"  Decode:      {pred_ps:.2f} tok/s")
+        print(f"  Prefill:     {prompt_n} tokens in {prompt_ms:.0f}ms"
+              f" ({prompt_n / max(prompt_ms, 1) * 1000:.1f} tok/s)")
+
+        results.append({
+            "config": name,
+            "spec_type": spec_type,
+            "predicted_tokens": pred_n,
+            "predicted_ms": pred_ms,
+            "predicted_per_second": pred_ps,
+            "prompt_tokens": prompt_n,
+            "prompt_ms": prompt_ms,
+            "wall_time": wall,
+        })
+
+        # Stop + mask between configs
+        local(f"sudo systemctl stop {SERVICE}")
+        local(f"sudo systemctl mask {SERVICE}")
+        time.sleep(2)
+
+    # Restore original SPEC_TYPE
+    sed_spec_type(orig_val)
+    print(f"\nRestoring SPEC_TYPE={orig_val}")
+    print("Unmasking + starting production service...")
+    local(f"sudo systemctl unmask {SERVICE}")
+    local(f"sudo systemctl start {SERVICE}")
+
+    # Health check
+    if wait_health(args.host, args.port):
+        print("Production service healthy.")
+    else:
+        print("WARNING: production service not healthy after restore!")
+
+    # Print results table
+    print(f"\n{'='*70}")
+    print("RESULTS")
+    print(f"{'='*70}")
+    print(f"{'Config':<30} {'Tokens':>8} {'Wall(s)':>10} {'Decode tok/s':>14} "
+          f"{'Error':>20}")
+    print(f"{'-'*30} {'-'*8} {'-'*10} {'-'*14} {'-'*20}")
+    for r in results:
+        if "error" in r:
+            print(f"{r['config']:<30} {'':>8} {'':>10} {'':>14} "
+                  f"{r['error']:>20}")
+        else:
+            print(f"{r['config']:<30} {r['predicted_tokens']:>8} "
+                  f"{r['wall_time']:>10.1f} {r['predicted_per_second']:>14.2f} "
+                  f"{'':>20}")
+    print(f"{'='*70}")
+
+    # Save JSON results
+    ts = time.strftime("%Y%m%d-%H%M%S")
+    out = f"/tmp/bench-spec-decode-{ts}.json"
+    with open(out, "w") as f:
+        json.dump({"results": results, "max_tokens": args.max_tokens,
+                   "timestamp": ts}, f, indent=2)
+    print(f"Results saved: {out}")
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+"""bench-vllm-corpus.py — build a code-shaped benchmark corpus from the repo.
+
+Why this exists: `vllm bench serve --dataset-name random` produces synthetic
+token soup, and speculative decoding accepts it nothing like real code — measured
+on frank-medium, random prompts accepted 2.9-3.6 tokens/update against the ~5.8
+the upstream reference reports on a code mix, which makes the random corpus
+under-report throughput by a wide margin and would pick the wrong SPEC depth.
+
+The prompts are the shape agents actually send: a repository artifact plus an
+instruction that requires reading it. Output length is fixed at bench time
+(`--custom-output-len`), so the corpus carries prompts only.
+
+Usage:
+    bench-vllm-corpus.py [repo-root] [out.jsonl]
+
+Then feed it to the endpoint (see scripts/bench-vllm-serve.sh, which passes the
+tokenizer the client needs and salts prompts so no run inherits a warm prefix
+cache). Methodology and the numbers these corpora produced are in
+serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md.
+"""
+
+from __future__ import annotations
+
+import json
+import pathlib
+import sys
+
+REPO = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else "/home/dank/serverAppConfigs")
+OUT = pathlib.Path(sys.argv[2] if len(sys.argv) > 2 else "/home/dank/bench/corpus-code.jsonl")
+
+TARGETS = [
+    "scripts/deploy-remote.sh",
+    "scripts/deploy-plan.py",
+    "scripts/deploy.sh",
+    "dankserver/frank-medium/radiance-entrypoint.sh",
+    "dankserver/frank-medium/quadlet/frank-medium-a.container",
+    "dankserver/frank-medium/proxy/affinity-proxy.py",
+    "dankserver/frank-medium/proxy/routing.py",
+    "tests/frank_medium_quadlet.bats",
+    "tests/deployment_scoping.bats",
+    "dankclaw/hermes/quadlet/hermes.container",
+    "dankclaw/anubis/quadlet/anubis.container",
+    "shared/prometheus/prometheus.yml",
+    "shared/prometheus/rules/llm-fleet.yml",
+    "shared/loki/loki-config.yaml",
+    "dankclaw/compose.yaml",
+    "dankserver/copyparty/quadlet/copyparty.container",
+]
+
+PROMPTS = [
+    "Read this repository artifact and summarise what it does, then list every "
+    "assumption it makes about the host it runs on.\n\n```\n{body}\n```",
+    "Here is a file from our homelab repository. Identify any bug, race or "
+    "missing error path, and show the minimal patch that fixes it.\n\n```\n{body}\n```",
+    "Explain this configuration to a new operator, then rewrite it with "
+    "comments that state the constraint each line protects.\n\n```\n{body}\n```",
+    "This file changed in a recent deploy. Write the changelog entry and a test "
+    "that would fail if the change regressed.\n\n```\n{body}\n```",
+]
+
+MAX_LINES = 220
+
+
+def main() -> int:
+    samples = []
+    for i, rel in enumerate(TARGETS):
+        path = REPO / rel
+        if not path.is_file():
+            continue
+        body = "\n".join(path.read_text(errors="replace").splitlines()[:MAX_LINES])
+        samples.append({"prompt": PROMPTS[i % len(PROMPTS)].format(body=body)})
+    if not samples:
+        print(f"no corpus sources found under {REPO}", file=sys.stderr)
+        return 1
+    OUT.parent.mkdir(parents=True, exist_ok=True)
+    with OUT.open("w") as fh:
+        for sample in samples:
+            fh.write(json.dumps(sample) + "\n")
+    print(f"wrote {len(samples)} prompts to {OUT}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,63 @@
+#!/bin/bash
+# bench-vllm-serve.sh — concurrency sweep against a vLLM endpoint.
+#
+# Usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]
+#   e.g. bench-vllm-serve.sh baseline 7770 1 4 8
+#
+# Runs `vllm bench serve` from inside the serving image (so the client matches
+# the server build) through an OpenAI-compatible endpoint, on the code corpus
+# scripts/bench-vllm-corpus.py writes.
+#
+# Two traps this script exists to encode, both of which produced silent garbage
+# during the 2026-09-16 frank-medium work:
+#
+#   1. The tokenizer must be passed explicitly and /models mounted. With only
+#      `--model frank-medium` the client tries to resolve that alias on
+#      HuggingFace and dies before sending a request — a run that looks like a
+#      benchmark and reports nothing.
+#   2. `--ignore-eos` is deliberate: it measures decode throughput at a fixed
+#      output length. Drop it when you care about real stop behaviour, but then
+#      do not compare the numbers with anything measured under it.
+#
+# Also note the corpora are code-shaped on purpose (see the generator's header):
+# synthetic random prompts misreport speculative acceptance badly.
+#
+# Methodology, and the numbers these runs produced on 2x R9700, are in
+# serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md.
+set -u
+
+TAG=${1:?usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]}
+PORT=${2:?usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]}
+shift 2
+CONCS=("$@")
+[ "${#CONCS[@]}" -eq 0 ] && CONCS=(1 4 8)
+
+IMG=${IMG:-localhost/radiance-vllm:0.9.3}
+MODELS_DIR=${MODELS_DIR:-/home/dank/models}
+TOK=${TOK:-/models/Qwen3.8-27B-Uncensored-MXFP4-awq}
+BENCH_DIR=${BENCH_DIR:-/home/dank/bench}
+CORPUS=${CORPUS:-$BENCH_DIR/corpus-code.jsonl}
+OUT=${OUT:-$BENCH_DIR/$TAG}
+mkdir -p "$OUT"
+
+if [ ! -f "$CORPUS" ]; then
+    echo "corpus $CORPUS is missing; run scripts/bench-vllm-corpus.py first" >&2
+    exit 1
+fi
+
+for c in "${CONCS[@]}"; do
+    echo "== $TAG conc=$c (port $PORT)"
+    podman run --rm --network=host -v "$MODELS_DIR":/models:ro -v "$BENCH_DIR":/bench:ro \
+        -e HF_HUB_OFFLINE=1 --entrypoint bash "$IMG" -lc "
+        vllm bench serve --backend openai-chat \
+          --base-url http://127.0.0.1:$PORT --endpoint /v1/chat/completions \
+          --model frank-medium --tokenizer $TOK \
+          --dataset-name custom --dataset-path /bench/$(basename "$CORPUS") \
+          --custom-output-len 512 --num-prompts 16 --max-concurrency $c \
+          --disable-shuffle --seed 0 --ignore-eos \
+          --percentile-metrics ttft,tpot --metric-percentiles 50,99 \
+          --disable-tqdm 2>&1 | grep -E 'Successful requests|Benchmark duration|Output token throughput|Total Token throughput|Mean TTFT|P50 TTFT|P50 TPOT|Acceptance rate|Acceptance length'
+    " 2>&1 | sed 's/^/  /'
+    echo
+done
+echo "results for $TAG written under $OUT"
@@ -0,0 +1,453 @@
+{%- set template_version = "qwen3.8-froggeric-v22.4.0" %}
+{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
+{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
+{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
+    {%- set _preserve_thinking = preserve_reasoning %}
+{%- elif preserve_thinking is defined and preserve_thinking is not none %}
+    {%- set _preserve_thinking = preserve_thinking %}
+{%- else %}
+    {%- set _preserve_thinking = true %}
+{%- endif %}
+{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
+{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
+{%- set _default_reasoning_effort = 'medium' %}
+{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
+{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}
+{%- set _initial_thinking = enable_thinking %}
+{%- if _effort_raw in ('none', 'off') %}
+    {%- set _initial_thinking = false %}
+    {%- set _initial_effort = 'medium' %}
+{%- elif _effort_raw in ('minimal', 'low') %}
+    {%- set _initial_effort = 'low' %}
+{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
+    {%- set _initial_effort = 'xhigh' %}
+{%- else %}
+    {%- set _initial_effort = 'medium' %}
+{%- endif %}
+{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
+{%- if auto_disable_thinking_with_tools and _has_tools %}
+    {%- set ns_state.thinking = false %}
+{%- endif %}
+{%- for msg in messages %}
+    {%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
+        {%- if msg.content is string %}
+            {%- if '<|think_off|>' in msg.content %}
+                {%- set ns_state.thinking = false %}
+            {%- elif '<|think_on|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+            {%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'xhigh' %}
+            {%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'low' %}
+            {%- elif '<|think_medium|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'medium' %}
+            {%- endif %}
+        {%- elif msg.content is iterable and msg.content is not mapping %}
+            {%- for item in msg.content %}
+                {%- if item is string %}
+                    {%- set _item_text = item %}
+                {%- elif item is mapping and 'text' in item and item.text is string %}
+                    {%- set _item_text = item.text %}
+                {%- else %}
+                    {%- set _item_text = '' %}
+                {%- endif %}
+                {%- if _item_text %}
+                    {%- if '<|think_off|>' in _item_text %}
+                        {%- set ns_state.thinking = false %}
+                    {%- elif '<|think_on|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                    {%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'xhigh' %}
+                    {%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'low' %}
+                    {%- elif '<|think_medium|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'medium' %}
+                    {%- endif %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set reasoning_instructions = '' %}
+{%- if ns_state.thinking %}
+    {%- if ns_state.effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif ns_state.effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if item is mapping %}
+                {%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain images.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set image_count.value = image_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+                {%- elif item.type == 'video' or 'video' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain videos.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set video_count.value = video_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Video ' ~ video_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+                {%- elif 'text' in item %}
+                    {{- item.text }}
+                {%- else %}
+                    {{- raise_exception('Unexpected item type in content.') }}
+                {%- endif %}
+            {%- else %}
+                {{- item | string }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set head = namespace(count=0, seen_non_system=false) %}
+{%- for message in messages %}
+    {%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
+    {%- if _is_sys and not head.seen_non_system %}
+        {%- set head.count = head.count + 1 %}
+    {%- else %}
+        {%- set head.seen_non_system = true %}
+    {%- endif %}
+{%- endfor %}
+{%- set sys_state = namespace(content='') %}
+{%- for message in messages[:head.count] %}
+    {%- set _part = render_content(message.content, false, true) | trim %}
+    {%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- if _part %}
+        {%- if sys_state.content %}
+            {%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
+        {%- else %}
+            {%- set sys_state.content = _part %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set _sc = sys_state.content %}
+{%- set _msgs = messages[head.count:] %}
+{%- if ns_state.thinking %}
+    {%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
+{%- else %}
+    {%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}
+{%- endif %}
+{%- set _terse_core %}
+Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
+Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
+If a user request is genuinely ambiguous, ask a sharp question, don't guess.
+{%- endset %}
+{%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}
+{#- `terse` chat-template kwarg: default true, so behaviour is unchanged unless a caller opts out.
+    Pass {"terse": false} via chat_template_kwargs to serve the model with only its own system
+    prompt -- useful for A/B-ing the terseness effect, or when a downstream prompt conflicts. #}
+{%- set _terse_on = terse if terse is defined else true %}
+{%- if _terse_on %}
+    {%- if not _sc %}
+        {%- set _sc = _terse | trim %}
+    {%- else %}
+        {%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}
+    {%- endif %}
+{%- endif %}
+{%- if _has_tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
+    {%- for tool in tools %}
+        {{- '\n' }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- '\n</tools>' }}
+    {%- if _tool_format == 'json' %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- else %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- endif %}
+    {%- if _sc %}
+        {{- '\n\n' + _sc }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if _sc %}
+        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set _last_idx = _msgs | length - 1 %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
+{%- for message in _msgs[::-1] %}
+    {%- set index = (_msgs | length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == 'user' %}
+        {%- set _rc = render_content(message.content, false) | trim %}
+        {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {%- if _last_idx > 50 %}
+        {%- set ns.last_query_index = _last_idx %}
+    {%- else %}
+        {%- set ns.last_query_index = 0 %}
+    {%- endif %}
+{%- endif %}
+{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
+{%- for message in _msgs %}
+    {%- set is_system = (message.role == "system" or message.role == "developer") %}
+    {%- set content = render_content(message.content, true, is_system) | trim %}
+    {%- if is_system or message.role == 'user' %}
+        {%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- endif %}
+    {%- if is_system %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'user' %}
+        {%- set ns2.consecutive_failures = 0 %}
+        {{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {%- set reasoning_content = '' %}
+        {%- set _explicit_reasoning = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- if message.reasoning_content is string %}
+                {%- set _explicit_reasoning = message.reasoning_content %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning_content | string %}
+            {%- endif %}
+        {%- elif message.thinking is defined and message.thinking is not none %}
+            {%- if message.thinking is string %}
+                {%- set _explicit_reasoning = message.thinking %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.thinking | string %}
+            {%- endif %}
+        {%- elif message.reasoning is defined and message.reasoning is not none %}
+            {%- if message.reasoning is string %}
+                {%- set _explicit_reasoning = message.reasoning %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning | string %}
+            {%- endif %}
+        {%- endif %}
+        {%- if _explicit_reasoning %}
+            {%- set _lead_end = '' %}
+            {%- if content.startswith('<think>') and '</think>' in content %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- elif content.startswith('</think>') %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _lead_end %}
+                {%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
+            {%- endif %}
+            {%- set reasoning_content = _explicit_reasoning %}
+        {%- else %}
+            {%- set _think_end = '' %}
+            {%- if content.startswith('</think>') %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _think_end = '</thinking>' %}
+            {%- elif '\n</think>' in content %}
+                {%- set _think_end = '\n</think>' %}
+            {%- elif '\n</thinking>' in content %}
+                {%- set _think_end = '\n</thinking>' %}
+            {%- elif '\n</ think>' in content %}
+                {%- set _think_end = '\n</ think>' %}
+            {%- elif '\n</think >' in content %}
+                {%- set _think_end = '\n</think >' %}
+            {%- elif content.startswith('<think>') and '</think>' in content %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _think_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _think_end %}
+                {%- if 'thinking' in _think_end %}
+                    {%- set _think_start = '<thinking>' %}
+                {%- else %}
+                    {%- set _think_start = '<think>' %}
+                {%- endif %}
+                {%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
+                {%- if _think_start in reasoning_content %}
+                    {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
+                {%- endif %}
+                {%- set content = content.split(_think_end)[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content | trim %}
+        {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>assistant\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined and tool_call.function is not none %}
+                    {%- set tc = tool_call.function %}
+                {%- else %}
+                    {%- set tc = tool_call %}
+                {%- endif %}
+                {%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
+                {%- if _tool_format == 'json' %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n' }}
+                    {%- endif %}
+                    {%- set _args = '{}' %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- elif tc.arguments is string %}
+                            {%- if tc.arguments %}
+                                {%- set _args = tc.arguments %}
+                            {%- endif %}
+                        {%- else %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
+                {%- else %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- else %}
+                            {{- '<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                    {%- endif %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- for args_name, args_value in tc.arguments.items() %}
+                                {{- '<parameter=' + args_name + '>\n' }}
+                                {%- if args_value is string %}
+                                    {%- set _av = args_value %}
+                                {%- else %}
+                                    {%- set _av = args_value | tojson %}
+                                {%- endif %}
+                                {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
+                                    {{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _av }}
+                                {%- endif %}
+                                {{- '\n</parameter>\n' }}
+                            {%- endfor %}
+                        {%- else %}
+                            {%- if tc.arguments is string %}
+                                {%- set _raw_args = tc.arguments %}
+                            {%- else %}
+                                {%- set _raw_args = tc.arguments | tojson %}
+                            {%- endif %}
+                            {%- if _raw_args %}
+                                {%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
+                                    {{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _raw_args }}
+                                {%- endif %}
+                            {%- endif %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '</function>\n</tool_call>' }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- set _content_lower = content | lower %}
+        {%- set _content_head = _content_lower[:120] %}
+        {%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
+        {%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
+        {%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
+        {%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
+        {%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
+        {%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
+        {%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
+            {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
+        {%- else %}
+            {%- set ns2.consecutive_failures = 0 %}
+        {%- endif %}
+        {%- if ns2.prev_role != 'tool' %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
+            {%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
+        {%- endif %}
+        {{- '\n<tool_response>\n' + content }}
+        {%- if ns2.consecutive_failures >= 2 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
+        {%- elif ns2.consecutive_failures == 1 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- set _next_role = _msgs[loop.index0 + 1].role %}
+            {%- if _next_role != 'tool' %}
+                {{- '<|im_end|>\n' }}
+            {%- endif %}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
+    {%- endif %}
+    {%- set ns2.prev_role = message.role %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if not ns_state.thinking %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}
@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+# Download the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT payload for
+# frank-medium from HuggingFace into
+# /home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT.
+#
+# Run on Dankserver (the box that serves it).  The repo is public, so the
+# download works anonymously; a token in ~/.cache/huggingface/token is used
+# when present, which keeps the ~28 GB main file clear of the anonymous
+# rate limit at full speed.
+#
+#   bash scripts/download-q8agent.sh
+#   bash scripts/download-q8agent.sh --verify-only
+#
+# The script is resumable (curl -C -) and verifies every artifact's size
+# afterwards, so a truncated file fails loudly instead of load-crashing.
+set -euo pipefail
+
+REPO="kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF"
+DIR="/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT"
+EXPECTED=(
+  "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf:28214425952"
+  "mmproj-Qwen3.8-27B-BF16.gguf:931145888"
+  "mtp-Qwen3.8-27B-Q4_0.gguf:1680271648"
+)
+# The model card's MTP prompt-cache fix ships in the same repo as a patch;
+# fetched for reference only.  This fork's main already carries the
+# speculative boundary in the context checkpoint, so nothing applies it.
+EXTRA=(
+  "patches/mtp-prompt-cache-fix.patch"
+)
+
+cd -- "$(dirname -- "$0")/.."
+
+main() {
+  local mode="download"
+  [[ "${1:-}" == "--verify-only" ]] && mode="verify"
+  mkdir -p "$DIR"
+  cd "$DIR"
+
+  local token="" auth=()
+  if [[ -r "$HOME/.cache/huggingface/token" ]]; then
+    token="$(cat "$HOME/.cache/huggingface/token")"
+    auth=(-H "Authorization: Bearer $token")
+  else
+    echo "no HF token at $HOME/.cache/huggingface/token; downloading anonymously"
+  fi
+
+  dl() {
+    local f="$1"
+    mkdir -p "$(dirname "$f")"
+    curl -fsSL -C - -o "$f" --retry 10 --retry-delay 5 --retry-max-time 10800 \
+      "${auth[@]}" \
+      "https://huggingface.co/$REPO/resolve/main/$f"
+    echo "  done: $f ($(du -h "$f" | cut -f1))"
+  }
+
+  if [[ "$mode" == "download" ]]; then
+    echo "downloading $REPO into $DIR"
+    for entry in "${EXPECTED[@]}"; do
+      dl "${entry%%:*}"
+    done
+    for f in "${EXTRA[@]}"; do
+      dl "$f"
+    done
+  fi
+
+  echo "verifying sizes"
+  local ok=1 f want act
+  for entry in "${EXPECTED[@]}"; do
+    f="${entry%%:*}"; want="${entry##*:}"
+    act="$(stat -c%s "$f" 2>/dev/null || echo 0)"
+    if [[ "$act" -eq "$want" ]]; then
+      echo "  OK   $f ($act bytes)"
+    else
+      echo "  BAD  $f ($act bytes, want $want)" >&2
+      ok=0
+    fi
+  done
+  if [[ -z "$token" ]]; then
+    echo "note: no token was used; sizes above are the integrity check"
+  fi
+  [[ "$ok" -eq 1 ]] || exit 1
+  echo "all files present and sized correctly"
+}
+
+main "$@"
@@ -0,0 +1,17 @@
+# Tunables for frank-default-npu (FastFlowLM NPU endpoint, Dankclaw)
+# Runtime: FLM 1.0.1 on the XDNA2 NPU (50 TOPS). Model: Gemma4-E4B-IT
+# (8B, Q4_1, vision + audio + reasoning). The NPU allocates from the
+# unified pool shared with frank-default-large (the CIRU llama.cpp
+# service on the iGPU). See serverAppConfigs-docs/FastFlowLM NPU Inference.md.
+#
+# 2026-08-20: gemma4-it:e4b deployed. Footprint ~9.1 GB. 64k context.
+
+FLM_MODEL=gemma4-it:e4b
+FLM_HOST=0.0.0.0
+FLM_PORT=52626
+FLM_PMODE=performance
+FLM_ASR=0
+# 64K context (native max for gemma4-it is 32768, but NPU supports up to 256k).
+# The model's max_prefill_len is 4096, so prompts longer than that get
+# truncated at prefill time.
+FLM_CTX_LEN=65536
@@ -0,0 +1,17 @@
+[Unit]
+Description=FastFlowLM NPU inference (frank-default-npu) on Dankclaw
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+Environment="FLM_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-npu.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-default-npu.sh
+LimitMEMLOCK=infinity
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,105 @@
+# Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
+# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
+# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
+#
+# Runtime: ik_llama.cpp master d85a1d7 (2026-09-15) with full qwen4exp
+# support (upstream PRs #2365/#2369/#2373 merged). Build recipe (CUDA for
+# the 3090 at sm_86, Zen5 AVX-512 IQK kernels for the CPU expert path):
+#   cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON \
+#     -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 \
+#     -DGGML_AVX512=ON -DGGML_AVX512_VBMI=ON -DGGML_AVX512_VNNI=ON \
+#     -DGGML_AVX512_BF16=ON -DLLAMA_CURL=OFF
+# The startup banner must print HAVE_FANCY_SIMD is defined — without those
+# macros the quantized GEMM silently falls back to AVX2 (no warning).
+#
+# Model: mradermacher imatrix quant of the uncensored Flash-Next (qwen4exp
+# MoE: 512 experts / 10 used, 48 layers = 12 attention + 36 GDN/ssm, QSA
+# indexer, hyper-connections, PLE n-gram table ~320M rows). 111 GiB shipped
+# as 3 parts, concatenated on disk; each part's sha256 was verified against
+# its HF LFS oid at download time.
+#
+# Placement (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
+#   - -ot "exps=CPU": ALL 512-expert tensors (121B elems, ~84 GB) live in
+#     the Strix Halo's LPDDR5X via the plain CPU backend buffer — mmap-backed
+#     page cache, ik's fastest known configuration for this shape (the
+#     author's 2x3090 + Threadripper reference box measures ~50 t/s on the
+#     same model family). The shared experts (shexp) keep their GPU-side
+#     default: they run on every token.
+#   - -ot "per_layer_token_embd=CPU": the ~32 GB PLE n-gram table follows —
+#     only a tiny slice is touched per token, so it streams from page cache.
+#   - NEVER use --cpu-moe on this host: it routes the CPU-overridden
+#     weights through the CUDA_Host PINNED buffer (measured 2026-09-15:
+#     108.12 GiB page-locked, evicted every co-tenant into swap and starved
+#     sshd into unreachability until a power cycle). -ot ...=CPU is the
+#     mmap-backed path with identical DDR5 bandwidth for resident pages,
+#     and the kernel degrades gracefully under pressure (page cache evict
+#     + re-read) instead of stranding anyone.
+#   - GGML_CUDA_NO_PINNED_WEIGHTS=1 (src/llama-load-tensors.cpp) keeps the
+#     CPU-overridden weights mmapped and leaves only the small staging
+#     buffers pinned. (GGML_CUDA_NO_PINNED would also unpin staging —
+#     unnecessary here and slower for prefill transfers.)
+#
+# Sampling: greedy-by-default to preserve the halogen-era client contract
+# (a request without temperature decodes greedy; temp 0.0 = argmax).
+# Clients that send their own temperature get their own values.
+#
+# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (see
+# the 2026-08-13 HIP incident in AGENTS.md). llama.cpp reads --host and
+# --temp as CLI flags, never as environment variables.
+
+# ik_llama.cpp runtime binary (master d85a1d7 build).
+LLAMA_BIN=/home/dank/GitPackages/ik_llama.cpp/build/bin/llama-server
+
+# Model package from
+# https://huggingface.co/mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF
+# (parts concatenated). The mmproj is the matching vision tower from the
+# static repo mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF — ik_llama.cpp
+# links examples/mtmd, so image input survives the runtime swap.
+MODEL_DIR=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M
+MODEL_FILE=Qwen3.8-Flash-Next-Uncensored.i1-Q4_K_M.gguf
+MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M/Qwen3.8-Flash-Next-Uncensored.mmproj-f16.gguf
+
+# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
+# server emits reasoning in DeepSeek format so clients receive
+# reasoning_content separately instead of literal think tags in content.
+CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
+
+# Alias + network.
+ALIAS=frank-dragoon
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770
+
+# Serving: single 262144-token slot (the model's full context window),
+# batch 4096 / ubatch 512, 16+16 threads (the Halo CPU has 16 cores, no
+# SMT), f16 KV. The KV cache scales with the 12 full-attention layers only
+# (~6 GB f16 at 262k); the GDN/ssm layers keep constant-size state. If
+# VRAM headroom is needed, --cache-type-k/v q8_0 halves the KV footprint.
+CTX_SIZE=262144
+PARALLEL=1
+BATCH_SIZE=4096
+UBATCH_SIZE=512
+NUM_THREADS=16
+THREAD_BATCH=16
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+
+# GPU/MoE placement: everything on the 3090 except the experts and the PLE
+# table, which both live in host RAM via the CPU backend buffer (see the
+# placement notes above).
+NGL=999
+# ik_llama.cpp CUDA host-memory policy: CPU-overridden weights stay mmapped;
+# only the small staging buffers are pinned (see the placement notes above).
+GGML_CUDA_NO_PINNED_WEIGHTS=1
+
+# Prompt cache (RAM budget in MiB) and slot state dir.
+CACHE_RAM=8192
+SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
+
+# Greedy-by-default (halogen contract). See sampling note above.
+TEMPERATURE=0.0
+
+# Reasoning on: DeepSeek format projects thinking into the API's
+# reasoning_content field instead of literal think tags in content;
+# unlimited reasoning budget (the model decides).
+REASONING_FORMAT=deepseek
+REASONING_BUDGET=-1
@@ -0,0 +1,21 @@
+[Unit]
+Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+# Staging buffers for the CUDA transfers are page-locked; the weights
+# themselves stay mmapped (GGML_CUDA_NO_PINNED_WEIGHTS=1), but the staging
+# budget still needs room. Parity with the halogen quadlet's
+# Ulimit=memlock=-1:-1.
+LimitMEMLOCK=infinity
+Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
				`@@ -1 +0,0 @@`
				`entropy UUID=cd86c728-459e-4f3a-bcc2-41ecf84685ec /etc/entropy nofail,luks`