feat(media): dedupe + relayout campaign tools for Audiobooks and eBooks

media-ab-reorg.py: planner for the ABS layout {Author}/{Series}/{Seq -
Year - Title {Narrator}} — claims one book folder per DB item (tracks
of duplicate recordings must not interleave), gates any book folder
that would nest above another (ABS's swallow defect), routes comic and
ebook-only items to intake, holds dedup losers under the campaign dir.
media-ab-verifydups.py: full-blake2b confirmation of size+head/tail
collision survivors. media-ebook-dedupe.py: dedupe the eBooks pile
against the Calibre library (identity key title-core+first-surname,
OPF metadata authority) and emit the calibredb ingest list.
abs-post-reorg.py: server-side ABS driver (cover export, DB-row wipe —
never hard=1 — scan, status) with credentials out of argv.

.AGENTTRASH/frank-default-small.env.llama.cpp

@@ -1,71 +0,0 @@
-# Tunables for frank-default-small (Tiel-Coder-35B-A3B UD-Q6_K_XL,
-# qwen35moe hybrid MoE, Dankserver, RTX 3090).
-#
-# Runtime: upstream llama.cpp (latest master, CUDA build with
-# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86) on RTX 3090
-# 24 GB; Ryzen 7 7700X; 62 GB RAM.  Replaced the ik_llama.cpp Ornith
-# CRACK Q8_0 deployment on 2026-09-01.
-#
-# Model: peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF, UD-Q6_K_XL quant
-# (30 GB).  Architecture: qwen35moe — 40 layers, full_attention_interval=4
-# (10 attention + 30 SSM/Mamba), 256 experts / 8 used, 2 KV heads × 256
-# dim, embedding 2048, rope dimension_count 64, rope freq_base 10 000 000.
-# Text-only (no mmproj).  Built-in chat template in the GGUF.
-#
-# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
-# Non-expert weights (~1.6 GB) fit entirely on the 24 GB card.
-#
-# KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per slot
-# × 3 parallel) and 10 attention layers, the KV cache is ~2 GB — the
-# requested 2 GB VRAM buffer.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
-# ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
-# incident). This box is NVIDIA/CUDA, but the rule keeps the env portable.
-
-# Model file (UD-Q6_K_XL, 30 GB, text-only MoE hybrid).
-MODEL_DIR=/home/dank/models/tiel-coder-35b
-MODEL_FILE=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
-
-# Upstream llama.cpp llama-server (CUDA build, latest master).
-# Built with -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86
-# using the staged CUDA 13.2 toolkit at /home/dank/cuda13-stage.
-LLAMA_SERVER=/home/dank/llama.cpp/build/bin/llama-server
-
-ALIAS=frank-default-small
-BIND_ADDR=0.0.0.0
-LISTEN_PORT=7770
-
-# Total context budget across all parallel slots.  200700 / 3 = 66 900
-# per slot.  The model's context_length is 262 144, so 66 900 is well
-# within range.
-CTX_SIZE=200700
-PARALLEL=3
-
-# Expert offload: all MoE expert weights pinned to CPU RAM.
-CPU_MOE=1
-
-# All non-expert layers on GPU (auto-fit will determine the exact count;
-# with --cpu-moe the non-expert footprint is ~1.6 GB, leaving ample room).
-N_GPU_LAYERS=999
-
-# KV cache: Q8_0 for both K and V.  ~2 GB total for 200700 ctx × 3 slots
-# × 10 attention layers × (2 KV heads × 256 dim × 1 byte) × 2 (K+V).
-CACHE_TYPE_K=q8_0
-CACHE_TYPE_V=q8_0
-
-VERBOSITY=2
-
-# Prefill batch sizing.
-BATCH_SIZE=4096
-UBATCH_SIZE=1024
-
-# Sampling.
-TEMPERATURE=1.0
-TOP_K=20
-TOP_P=0.95
-MIN_P=0.0
-
-# Compute threads lowered so other processes keep a core.
-THREADS=7
-THREADS_HTTP=8

.AGENTTRASH/frank-default-small.service.llama.cpp

@@ -1,16 +0,0 @@
-[Unit]
-Description=llama.cpp Tiel-Coder-35B-A3B Q6_K_XL qwen35moe hybrid MoE (dankserver, frank-default-small)
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-small.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

.AGENTTRASH/frank-dragoon.env.llama.cpp

@@ -1,126 +0,0 @@
-# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
-# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
-# Strix Halo gfx1151, unified memory).
-#
-# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
-# qwen4exp-on-rocmfpx-d3ca537 patch applied — the model card's pinned
-# runtime: ROCmFP4 tensor types AND the qwen4exp architecture in one
-# tree; stock llama.cpp and charlie's fork alone cannot load this GGUF).
-# Built per the card recipe:
-#   cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
-# at /home/dank/GitPackages/kingjones-ROCmFPX/build.
-# Replaced the agentionai ROCmFP4-FAST imatrix deployment (charlie's
-# build-qwen38 runtime) on 2026-08-30.
-#
-# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
-# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
-# at layer 1), Q4_0_ROCMFP4_STRIX (file_type 105), 113.47 GiB single
-# file. STRIX is the top quality tier: PLE table at Q8_0 (50.66 GiB),
-# token embeddings + lm head at Q6_K. The PLE table NEVER enters GPU
-# memory — it streams from the file through the OS page cache. Two hard
-# card constraints:
-#  1. mmap MUST stay enabled (default) — --no-mmap forces the 50.66 GiB
-#     table into anonymous memory and the cgroup killer takes the
-#     process with nothing in the server log (only dmesg).
-#  2. Do NOT pin the PLE tensor to CPU with -ot — measured gen fell
-#     ~23 -> 13.4 tok/s; the kernel streams it better.
-# NOTE: the card's own build_attn_qsa still calls ggml_fill on the
-# F16-cast kq mask (the path that aborted our charlie build with
-# --flash-attn on before scripts/patches/qwen4exp-qsa-f32-mask.patch).
-# If a clean kingjones-tree build aborts graph build with
-# GGML_ASSERT(a-type == GGML_TYPE_F32), port that fix (F32 base mask +
-# cast at the attention call) into this tree and rebuild.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
-# incident). llama.cpp reads --host/--port/--temp as CLI flags, never as
-# env vars.
-
-# kingjones30/ROCmFPX runtime binary (card recipe build).
-LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
-
-# Model package from
-# https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
-# Layout: single unsharded GGUF, text-only (the repo ships no mmproj;
-# the vision tower below is borrowed from the agentionai FAST package —
-# same base model, f16 — kept only if it loads clean, drop
-# MMPROJ_PATH if it does not).
-MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
-MODEL_FILE=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
-MMPROJ_PATH=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf
-
-# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
-# Sharp template drives thinking on by default (medium effort); the server
-# emits reasoning in DeepSeek format so clients receive reasoning_content
-# separately instead of literal thinking tags in content.
-CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
-CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
-
-# Alias + network.
-ALIAS=frank-dragoon
-BIND_ADDR=0.0.0.0
-LISTEN_PORT=7770
-
-# Serving: 401408-token context, TWO slots (parallel 2), batch 4096 /
-# ubatch 512, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
-# prompt cache. --fit off: auto-fit segfaults on this model's
-# hyper-connection/indexer config; -ngl 999 puts every layer on the
-# SVM-backed GPU pool. ubatch 512 matches the card's re-measurement
-# (-ub 1024/2048/4096 showed no effect; -t 32 vs 16 is noise).
-#
-# CTX_SIZE=401408 / PARALLEL=2: carried over from the FAST deployment
-# where it loaded clean (~74 s, health 200). The card measures the full
-# 262144 window at ~71.7 GiB GTT at load on the same 128 GB board; QSA
-# caps KV growth so the extra window is cheap. KV footprint scales
-# with the 12 attention layers only. Verify load on first STRIX boot.
-CTX_SIZE=401408
-PARALLEL=2
-
-BATCH_SIZE=4096
-UBATCH_SIZE=512
-NUM_THREADS=16
-THREAD_BATCH=16
-CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
-CTX_CHECKPOINTS=16
-CACHE_RAM=8192
-SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
-SPLIT_MODE=none
-FIT=off
-NGL=999
-FLASH_ATTN=on
-
-# Sampling (card instruct table, non-thinking).
-TEMPERATURE=0.7
-TOP_K=20
-TOP_P=0.8
-MIN_P=0.0
-PRESENCE_PENALTY=1.5
-
-# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
-# keep the reasoning trail in the conversation). DeepSeek format projects the
-# reasoning into the API's reasoning_content field instead of literal thinking
-# tags.
-REASONING=on
-REASONING_FORMAT=deepseek
-
-# Strix Halo unified memory + native gfx1151.
-HSA_OVERRIDE_GFX_VERSION=11.5.1
-
-VERBOSITY=4
-
-
-# Speculative decoding: disabled. The card's Usage block ships no spec
-# flags. The Flash-Next MTP heads do not load on this loader (requires
-# per-layer blk.N.hc_attn_norm; every head carries only blk.48 —
-# verified 3x on build-qwen38). DRAFT_MODEL_* point at the retired
-# agentionai FAST head and are unused while SPEC_TYPE=none.
-DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
-DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
-SPEC_TYPE=none
-SPEC_DRAFT_N_MAX=3
-SPEC_DRAFT_N_MIN=0
-SPEC_DRAFT_P_MIN=0.75
-SPEC_DRAFT_NGL=999
-SPEC_DRAFT_DEVICE=ROCm0
-SPEC_DRAFT_TYPE_K=f16
-SPEC_DRAFT_TYPE_V=f16

.AGENTTRASH/frank-dragoon.service.llama.cpp

@@ -1,16 +0,0 @@
-[Unit]
-Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

.AGENTTRASH/frank-npu.env.flm-npu

@@ -1,23 +0,0 @@
-# Tunables for frank-npu (FastFlowLM NPU endpoint, Dankclaw)
-# Runtime: FLM 1.0.1 (/usr/bin/flm) on the XDNA2 NPU (50 TOPS). Boot tag is
-# nuextract3:4b (NuExtract3-4B-NPU2, Qwen3.5-4B skeleton, structured-extraction
-# model); serve routes every request by the exact FLM registry tag in the
-# request's model field, so any registered tag (gemma4-it:e4b, qwythos:9b, ...)
-# is reachable on this port regardless of the boot tag. The NPU is a
-# single-lane device: generations serialize FIFO, and the first request after a
-# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
-# pool shared with frank-dragoon (iGPU).
-FLM_MODEL=nuextract3:4b
-FLM_HOST=0.0.0.0
-FLM_PORT=7771
-# turbo = max NPU power mode (xrt-smi configure --pmode turbo at serve start).
-# No other tenant uses the NPU on this host, so the top mode costs nothing to
-# neighbours; drop to `performance` if iGPU power-budget contention ever shows.
-FLM_PMODE=turbo
-# FLM_ASR=1 additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes
-# /v1/audio/transcriptions. NuExtract3 answers text-only; audio stays off.
-FLM_ASR=0
-# -1 = no --ctx-len flag: every pack loads with its own trained context
-# (nuextract3:4b -> 32768, gemma4-it:e4b -> 131072). A global override would
-# over-allocate KV for the small packs.
-FLM_CTX_LEN=-1

.AGENTTRASH/frank-npu.service.flm-npu

@@ -1,17 +0,0 @@
-[Unit]
-Description=FastFlowLM NPU sight+sound inference (frank-npu) on Dankclaw
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-Environment="FRANK_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-npu.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-npu.sh
-LimitMEMLOCK=infinity
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

.AGENTTRASH/serve-frank-dragoon.sh.llama.cpp

@@ -1,180 +0,0 @@
-#!/usr/bin/env bash
-# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
-# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
-# Dankclaw (Strix Halo gfx1151, unified memory).
-#
-# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
-# model card's qwen4exp-on-rocmfpx patch) built at
-# /home/dank/GitPackages/kingjones-ROCmFPX/build (HIP, gfx1151,
-# ROCm 7.2.4, card recipe). The ROCmFP4 quant types plus the qwen4exp
-# architecture must be in one tree; stock llama.cpp cannot load this
-# GGUF.
-#
-# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
-# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
-# at layer 1) quantized Q4_0_ROCMFP4_STRIX, 113.47 GiB single file. The
-# 50.66 GiB PLE n-gram table stays OUT of GPU memory and streams via
-# mmap from the OS page cache — never add --no-mmap or -ot for it.
-#
-# Profile: card sampling table (temp 0.7, top_p 0.80, top_k 20, min_p 0.0,
-# presence 1.5 — non-thinking instruct) mixed with the validated serving
-# profile: ctx 401408 x 2 slots, b 4096 / ub 512, t 16 / tb 16, f16 KV,
-# fa on (if the graph build aborts on the QSA F16 mask, see the note in
-# the env file: port scripts/patches/qwen4exp-qsa-f32-mask.patch into
-# this tree and rebuild), cont-batching, prompt cache + 8 GiB cache-ram
-# + idle-slot cache, 16 ctx checkpoints, --fit off (auto-fit segfaults
-# on qwen4exp hyper-connection config). Speculative decoding off: the
-# Flash-Next MTP heads do not load on this loader; the card ships no
-# spec flags.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -------------------
-: "${LLAMA_BIN:=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
-: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
-: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
-: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
-: "${CHAT_TEMPLATE_KWARGS:={}}"
-: "${ALIAS:=frank-dragoon}"
-: "${BIND_ADDR:=0.0.0.0}"
-: "${LISTEN_PORT:=7770}"
-: "${CTX_SIZE:=401408}"
-: "${PARALLEL:=2}"
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=512}"
-: "${NUM_THREADS:=16}"
-: "${THREAD_BATCH:=16}"
-: "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-: "${CTX_CHECKPOINTS:=16}"
-: "${CACHE_RAM:=8192}"
-: "${SLOT_SAVE_PATH:=$SCRIPT_DIR/slot-state}"
-: "${SPLIT_MODE:=none}"
-: "${FIT:=off}"
-: "${NGL:=999}"
-: "${FLASH_ATTN:=on}"
-: "${TEMPERATURE:=0.7}"
-: "${TOP_K:=20}"
-: "${TOP_P:=0.8}"
-: "${MIN_P:=0.0}"
-: "${PRESENCE_PENALTY:=1.5}"
-: "${REASONING:=on}"
-: "${REASONING_FORMAT:=deepseek}"
-: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
-: "${SPEC_TYPE:=none}"
-: "${DRAFT_MODEL_DIR:=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST}"
-: "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
-: "${SPEC_DRAFT_N_MAX:=3}"
-: "${SPEC_DRAFT_N_MIN:=0}"
-: "${SPEC_DRAFT_P_MIN:=0.75}"
-: "${SPEC_DRAFT_NGL:=999}"
-: "${SPEC_DRAFT_DEVICE:=ROCm0}"
-: "${SPEC_DRAFT_TYPE_K:=f16}"
-: "${SPEC_DRAFT_TYPE_V:=f16}"
-
-MODEL="$MODEL_DIR/$MODEL_FILE"
-for f in "$LLAMA_BIN" "$MODEL" "$CHAT_TEMPLATE_FILE"; do
-    if [[ ! -r "$f" ]]; then
-        echo "missing or unreadable required artifact: $f" >&2
-        exit 1
-    fi
-done
-if [[ ! -x "$LLAMA_BIN" ]]; then
-    echo "missing executable runtime: $LLAMA_BIN" >&2
-    exit 1
-fi
-
-# --- speculative decoding (MTP draft head) ---------------------------------
-# The MTP draft head is a separate GGUF sidecar loaded via --spec-draft-model
-# (alias --model-draft).  ngram-only specs do not need a draft file.
-# Set SPEC_TYPE=none in the env to disable speculative decoding entirely.
-SPEC_ARGS=()
-if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
-    SPEC_ARGS=(
-        --spec-type "$SPEC_TYPE"
-        --spec-draft-n-max "$SPEC_DRAFT_N_MAX"
-        --spec-draft-n-min "$SPEC_DRAFT_N_MIN"
-        --spec-draft-p-min "$SPEC_DRAFT_P_MIN"
-    )
-    if [[ "$SPEC_TYPE" == *draft-mtp* ]]; then
-        DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
-        if [[ ! -r "$DRAFT_MODEL" ]]; then
-            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
-            exit 1
-        fi
-        SPEC_ARGS+=(
-            --spec-draft-model "$DRAFT_MODEL"
-            --spec-draft-ngl "$SPEC_DRAFT_NGL"
-            --spec-draft-device "$SPEC_DRAFT_DEVICE"
-            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
-            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
-        )
-    fi
-    if [[ -n "${SPEC_DRAFT_P_SPLIT:-}" ]]; then
-        SPEC_ARGS+=(--spec-draft-p-split "$SPEC_DRAFT_P_SPLIT")
-    fi
-fi
-
-# The STRIX repo ships no mmproj; this is the agentionai FAST package's
-# f16 vision tower for the same base model. Passed only when readable.
-MMPROJ_ARGS=()
-if [[ -n "${MMPROJ_PATH:-}" && -r "${MMPROJ_PATH:-}" ]]; then
-    MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
-fi
-
-SERVER_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-mkdir -p "$SLOT_SAVE_PATH"
-ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
-LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
-
-exec /usr/bin/env \
-    ROCM_PATH="$ROCM_PATH" \
-    HIP_VISIBLE_DEVICES=0 \
-    LD_LIBRARY_PATH="$LIBRARY_PATH" \
-    HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
-    "$LLAMA_BIN" \
-    --model "$MODEL" \
-    --alias "$ALIAS" \
-    --chat-template-file "$CHAT_TEMPLATE_FILE" \
-    --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --jinja \
-    --device ROCm0 \
-    --gpu-layers "$NGL" \
-    --split-mode "$SPLIT_MODE" \
-    --parallel "$PARALLEL" \
-    --ctx-size "$CTX_SIZE" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --fit "$FIT" \
-    --flash-attn "$FLASH_ATTN" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --threads "$NUM_THREADS" \
-    --threads-batch "$THREAD_BATCH" \
-    --ctx-checkpoints "$CTX_CHECKPOINTS" \
-    --cache-ram "$CACHE_RAM" \
-    --slot-save-path "$SLOT_SAVE_PATH" \
-    --cache-prompt \
-    --cache-idle-slots \
-    --metrics \
-    --temp "$TEMPERATURE" \
-    --top-p "$TOP_P" \
-    --top-k "$TOP_K" \
-    --min-p "$MIN_P" \
-    --presence-penalty "$PRESENCE_PENALTY" \
-    --reasoning "$REASONING" \
-    --reasoning-format "$REASONING_FORMAT" \
-    --reasoning-budget -1 \
-    "${MMPROJ_ARGS[@]}" \
-    "${SPEC_ARGS[@]}"

.AGENTTRASH/serve-frank-npu.sh.flm-npu

@@ -1,62 +0,0 @@
-#!/usr/bin/env bash
-# Serve the FastFlowLM (FLM) sight+sound endpoint — frank-npu.
-#
-# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
-# single-lane device: generations serialize in FIFO order, so this is a
-# background/triage endpoint, not an interactive multi-user chat. The NPU
-# draws from the unified RAM pool shared with frank-dragoon (the iGPU
-# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
-# for the memory budget and the model-selection tradeoff.
-#
-# NOTE: FLM requires unlimited memlock. The unit sets LimitMEMLOCK=infinity.
-# NOTE: an unknown model tag does NOT 404 — /v1/chat/completions answers HTTP 200 with a
-# flat {"error":"Failed to load <tag> model!"} body and no choices; /v1/completions returns a
-# nested OpenAI-shaped server_error (code 500) also under HTTP 200. Client model IDs must be the exact FLM registry
-# tag (e.g. gemma4-it:e4b), never a service alias. Custom tags only resolve
-# when FLM_CONFIG_PATH points at the user registry that carries them and their
-# kernels are symlinked into /home/dank/.config/flm/xclbins/<PackName>.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_NPU_ENV_FILE:-$SCRIPT_DIR/frank-npu.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -------------------
-: "${FLM_BIN:=/usr/bin/flm}"
-: "${FLM_MODEL:=gemma4-it:e4b}"
-: "${FLM_HOST:=0.0.0.0}"
-: "${FLM_PORT:=7771}"
-: "${FLM_PMODE:=turbo}"
-: "${FLM_ASR:=0}"
-: "${FLM_CTX_LEN:=131072}"
-
-# The user registry is a superset of the system registry (it carries the
-# custom tags too), and its model_path resolves relative to the registry file,
-# so exports keep a tag swap in the env file from needing script edits.
-export FLM_CONFIG_PATH="${FLM_CONFIG_PATH:-/home/dank/.config/flm/model_list.json}"
-export FLM_XCLBIN_PATH="${FLM_XCLBIN_PATH:-/home/dank/.config/flm/xclbins}"
-
-if [[ ! -x "$FLM_BIN" ]]; then
-    echo "missing flm binary: $FLM_BIN" >&2
-    exit 1
-fi
-
-# --- optional flags ----------------------------------------------------------
-OPT_ARGS=()
-if [[ "${FLM_ASR}" != "0" ]]; then
-    OPT_ARGS+=(--asr "${FLM_ASR}")
-fi
-if [[ "${FLM_CTX_LEN}" != "-1" ]]; then
-    OPT_ARGS+=(--ctx-len "${FLM_CTX_LEN}")
-fi
-
-exec "$FLM_BIN" serve "$FLM_MODEL" \
-    --host "$FLM_HOST" \
-    -p "$FLM_PORT" \
-    --pmode "$FLM_PMODE" \
-    "${OPT_ARGS[@]}"

.AGENTTRASH/serve-frank-small.sh.llama.cpp

@@ -1,87 +0,0 @@
-#!/usr/bin/env bash
-# Serve peculiar-ragdoll/Tiel-Coder-35B-A3B (UD-Q6_K_XL) as
-# frank-default-small via stock llama.cpp on Dankserver (RTX 3090).
-#
-# Runtime: upstream llama.cpp (latest master, CUDA build with
-# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86).
-# The model is a 35B-total / 8-active MoE hybrid (qwen35moe arch) with
-# SSM/Mamba layers (full_attention_interval=4: 10 attention + 30 SSM
-# layers out of 40).  256 experts, 8 used per token.
-#
-# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
-# Non-expert weights (attention, SSM, shared expert, embeddings, output)
-# are ~1.6 GB and fit entirely on the 24 GB RTX 3090.
-#
-# KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per
-# slot × 3 parallel) and 10 attention layers (2 KV heads × 256 dim),
-# the KV cache is ~2 GB — the requested 2 GB VRAM buffer.
-#
-# Model is loaded with mmap (default).  The 30 GB Q6_K_XL file is
-# memory-mapped; only accessed pages enter RAM.
-#
-# Tunables come from frank-default-small.env unless FRANK_SMALL_ENV_FILE
-# overrides that path.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_SMALL_ENV_FILE:-$SCRIPT_DIR/frank-default-small.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -----------------
-: "${MODEL_DIR:=/home/dank/models/tiel-coder-35b}"
-: "${MODEL_FILE:=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf}"
-: "${LLAMA_SERVER:=/home/dank/llama.cpp/build/bin/llama-server}"
-: "${ALIAS:=frank-default-small}"
-: "${BIND_ADDR:=0.0.0.0}"
-: "${LISTEN_PORT:=7770}"
-: "${CTX_SIZE:=200700}"
-: "${PARALLEL:=3}"
-: "${CACHE_TYPE_K:=q8_0}"
-: "${CACHE_TYPE_V:=q8_0}"
-: "${N_GPU_LAYERS:=999}"
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=1024}"
-: "${VERBOSITY:=2}"
-: "${CPU_MOE:=1}"
-: "${TEMPERATURE:=1.0}"
-: "${TOP_K:=20}"
-: "${TOP_P:=0.95}"
-: "${MIN_P:=0.0}"
-: "${THREADS:=7}"
-: "${THREADS_HTTP:=8}"
-
-MODEL="$MODEL_DIR/$MODEL_FILE"
-
-if [[ ! -f "$MODEL" ]]; then
-    echo "missing required file: $MODEL" >&2
-    exit 1
-fi
-if [[ ! -x "$LLAMA_SERVER" ]]; then
-    echo "missing llama-server binary: $LLAMA_SERVER" >&2
-    exit 1
-fi
-
-exec "$LLAMA_SERVER" \
-    --model "$MODEL" \
-    $([ "$CPU_MOE" = "1" ] && printf -- "--cpu-moe") \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --ctx-size "$CTX_SIZE" \
-    --parallel "$PARALLEL" \
-    --gpu-layers "$N_GPU_LAYERS" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --temp "$TEMPERATURE" \
-    --top-k "$TOP_K" \
-    --top-p "$TOP_P" \
-    --min-p "$MIN_P" \
-    --threads "$THREADS" \
-    --threads-http "$THREADS_HTTP" \
-    --alias "$ALIAS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT"

AGENTS.md

@@ -156,6 +156,15 @@ Unit requirements:
   (`<host>/<svc>/quadlet/`), is installed by the deployer, and is covered by
   the quadlet test seam. The host-side unit is generated state; the repo file
   is the source of truth.
+- **Bind-source gate** — a unit whose `Volume=` source is a mount this host owns
+  (the entropy array, the metadata view) carries
+  `ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh <source>`.
+  Measured 2026-10-31: in a **rootless** unit `RequiresMountsFor=` is inert (the
+  user manager cannot order against the system's mount unit), and
+  `ConditionPathIsMountPoint=` is fail-*dead* — an unmet condition logs `skipped,
+  unmet condition check` with `NRestarts=0` and no start attempt ever follows.
+  Bind the array before it is mounted and the container sees the empty pre-mount
+  directory as truth; a media scan then deletes the library it cannot see.
 
 Before declaring a service deployed: `systemctl --user is-enabled <name>` →
 `enabled`, `is-active` → `active`, and the container answers its health check.
@@ -218,9 +227,28 @@ See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
 
 ## External Exposure Policy (MANDATORY)
 
-Every Caddy-proxied service is either behind the auth chain (Anubis :8923 →
-Authentik embedded outpost) or explicitly internal-only — no public DNS name,
-no vhost. New services default to internal-only until guarded.
+Every Caddy-proxied service is in exactly one of three postures: behind the
+auth chain (Anubis :8923 → Authentik embedded outpost), **exempt by
+construction** because it authenticates itself (`auth.`, git smart-HTTP, Gitea,
+OpenWebUI, Jellyfin, Plex, Audiobookshelf, the *arr stack), or internal-only —
+served to nobody off-LAN. New services default to internal-only until guarded.
+
+**Internal-only is a source gate, not a missing DNS record.** Caddy answers
+every Host it knows, and its per-name certificates are published in Certificate
+Transparency logs, so a name with no Porkbun A record is still in a stranger's
+reach (`crt.sh` lists it). A household-only service therefore carries an edge
+gate — `@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8
+172.16.0.0/12`, `handle @lan { reverse_proxy … }`, `handle { respond 404 }` —
+which is what the *arr stack and qBittorrent carry (2026-11-04: Frank's law,
+"never leaving the home boxes"). A future household-only vhost gets the same
+block, not merely an absent record.
+
+The guard chain is a **browser-only device**: Anubis demands a JavaScript
+proof-of-work and the outpost works from a session cookie, so any non-browser
+client — a native app, a podcast RSS reader, an OPDS client, `git`, a
+service-to-service call — is answered with challenge HTML where it expected
+JSON or a socket payload. A service that must serve such a client cannot sit
+behind the chain; it has to own its authentication.
 
 When guarding a new vhost — provider first, then route:
 
@@ -239,6 +267,10 @@ When guarding a new vhost — provider first, then route:
 Exempt categories (document the reason in the vhost comment): the IdP itself
 (`auth.`), git smart-HTTP endpoints, services with their own complete auth.
 
+Unguarding a vhost is the mirror image of the procedure above — the route flips
+first and the IdP rows go last — and `serverAppConfigs-docs/Infrastructure/Caddy.md`
+*Unguard a vhost* owns the steps.
+
 ## Deletion & Trash Policy (MANDATORY)
 
 Never `rm`. Move deletions into trash so they stay recoverable:
@@ -256,7 +288,10 @@ archive.
 **Swap the model. Do not instrument it.** A payload change is: edit the trio
 (env / serve script / unit) → `scripts/deploy.sh <host> --service <svc>` →
 `/health` answers → one chat turn, one tool call, and one vision probe on an
-endpoint that serves images. That is the entire procedure.
+endpoint that serves images → re-key that endpoint's OpenWebUI `model` row
+(`meta.description` + `params.context.length`, API door in [[OpenWebUI]]). That is
+the entire procedure — no deployer step touches `webui.db`, and a stale
+`context.length` makes OpenWebUI advertise a window the endpoint does not grant.
 
 - **No benchmarking unless Frank orders it.** No pre-swap baseline battery,
   no A/B arms, no decode/prefill ladders, no acceptance tables, no numbers
@@ -467,8 +502,8 @@ disagrees with it is the bug.
   `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
   `scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
   `stilldeadcode/radiance@sha256:d590e300…` (radiance 1.8.1); container name = unit name.
-- Shape: TP=2 over `wht6`, `--max-num-seqs 2`, `--max-model-len 200000`,
-  `expert_tiered` placement, 44,928 MiB pinned host pool per rank, `fp8` KV,
+- Shape: TP=2 over `wht6`, `--max-num-seqs 4`, `--max-model-len 200000`,
+  `expert_tiered` placement, 49,152 MiB pinned host pool per rank, `fp8` KV,
   MTP depth 2, MNBT 2048, `--busy-prefill-chunk auto`, prefix cache 8 GiB host
   tier only, n-gram table pinned to RAM. The per-card weights/KV split is derived
   by the engine's budget resolver from `--gpu-headroom-mib`, the cache bound and
@@ -726,7 +761,12 @@ linked runbooks.
 **Ops**
 
 - Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
-  Use `pkill -x <exact-name>`.
+  Use `pkill -x <exact-name>`, or the bracket trick — `pkill -f "serve[-]script.sh"` —
+  when the pattern has to be named: the bracket makes the pattern stop matching the
+  command line that carries it. Quote it: a zsh remote glob-expands an unquoted `[s]`
+  and the kill silently never runs. Re-hit 2026-11-02 —
+  `ssh host 'pkill -f jfshowswatch.py; …'` killed the remote shell mid-command, and
+  ssh exits **255 with zero output**, which reads exactly like a network fault.
 - Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
   (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
 - **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
@@ -808,6 +848,8 @@ per-attempt page and no per-swap ledger anywhere else.
 | Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
 | Movie-library relayout (Jellyfin format, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Relayout.md` |
+| SSD-backed metadata view (Jellyfin writes, Plex reads, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Metadata View.md` |
+| Duplicate films and the purge hold (Dankserver) | `serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md` |
 
 There is no page for a retired endpoint and no change diary. A payload swap
 appends one row to the live endpoint's **Eras** table; when the endpoint itself
@@ -850,5 +892,6 @@ history.
 | Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
 | Relayout the movie library | **Census → plan → read the exceptions → `scripts/media-apply.py`** — renames only, journal before every move; never hand-rename, never copy. |
+| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. **Deployed on Dankserver 2026-10-31, re-verified 2026-11-01: both quadlets bind the view, not the raw array. A restart for any reason discards that server's in-flight library scan, so re-trigger the refresh — see [[Media Metadata View]].** |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |

dankclaw/Caddyfile

@@ -22,13 +22,16 @@
 # CHALLENGE everyone else) → Caddy's internal listener http://:8016 (auth
 # decision; Host headers survive every hop) → authenticated traffic is
 # forwarded back to http://:8017 and routed straight to the app.
-# Apps with their own auth (Gitea, OpenWebUI, Jellyfin, *arr, Pi-hole) and the
-# Authentik IdP itself (auth.franksdank.cloud — its login flow must never sit
-# behind a challenge) are exempt by construction: their vhosts never route
-# through Anubis. ComfyUI left this exempt set on 2026-08-31 and Grafana left
-# it on 2026-09-01 (OAuth2 relying-party mode against authentik): both now sit
-# behind the guard chain; Grafana authenticates users itself via the OAuth2
-# code flow, so no identity header is forwarded to it.
+# Apps with their own auth (Gitea, OpenWebUI, Jellyfin, Plex, Audiobookshelf,
+# *arr, Pi-hole) and the Authentik IdP itself (auth.franksdank.cloud — its login
+# flow must never sit behind a challenge) are exempt by construction: their
+# vhosts never route through Anubis. The chain is browser-only by nature — a
+# native app client, a podcast RSS reader or a git client solves no JavaScript
+# proof, so anything that must serve one has to own its own authentication.
+# ComfyUI left this exempt set on 2026-08-31 and Grafana left it on 2026-09-01
+# (OAuth2 relying-party mode against authentik): both now sit behind the guard
+# chain; Grafana authenticates users itself via the OAuth2 code flow, so no
+# identity header is forwarded to it.
 # The pihole vhost keeps its basicauth edge; the hash placeholder below is
 # rendered by deploy.sh from CADDY_BASICAUTH_HASH (host .env, git-ignored) into
 # /home/dank/Containers/caddy/Caddyfile, which the container mounts.
@@ -264,20 +267,67 @@ files.franksdank.cloud {
 	}
 }
 
-# --- Jellyfin ---
+# --- Audiobookshelf (exempt by construction: own auth) --------------------
+# Caddy → app: no Anubis, no outpost. The guard chain answers a native client's
+# /ping, /api/login and /socket.io/ handshake with challenge HTML, so behind it
+# the phone app parsed a non-object where the socket `init` payload belongs and
+# podcast RSS/OPDS feeds broke the same way. ABS authenticates itself (local
+# accounts plus authentik OIDC), so it joins jelly/plex/*arr outside the chain
+# (2026-11-03); the IdP rows that guarded this name — proxy provider
+# `audiobooks-proxy` and application "Audiobookshelf Guard" — are retired with
+# the change, after the route flips. Accepted exposure: /audiobookshelf/api/login
+# is internet-visible with no rate limiter on either side (this Caddy build has
+# no rate_limit module; ABS 2.37.2 ships no lockout) — the same class every other
+# own-auth vhost here lives with, so shared accounts need long unique passwords.
+# ROUTER_BASE_PATH=/audiobookshelf, so the bare root is a convenience hop onto it.
+# X-Forwarded-Proto must say https: ABS derives its OIDC origin from
+# x-forwarded-proto://Host and this hop is plain http, so without the pin its own
+# https callback is a 400 "Invalid callback URL - must be same-origin".
+audiobooks.franksdank.cloud {
+	redir / /audiobookshelf/ permanent
+	reverse_proxy 192.168.0.235:8000 {
+		header_up X-Forwarded-Proto https
+	}
+}
+
+# --- Jellyfin (exempt by construction: own auth) --------------------------
+# No Anubis, no outpost: Jellyfin carries its own complete login (its own
+# account store, no Authentik link), which is the policy's exempt category —
+# the same family as plex/ABS/*arr around it. Accepted exposure: the sign-in
+# form is internet-visible with no rate limiter on either side (this Caddy
+# build ships no rate_limit module), so accounts need long unique passwords.
+# The homepage card opened it to the household on 2026-11-03; accounts are
+# created in Jellyfin's own admin UI, not by SSO.
 jelly.franksdank.cloud {
 	reverse_proxy 192.168.0.235:8096
 }
 
 
-# --- *arr Stack (exempt by construction: own auth) ---
-
+# --- *arr Stack + qBittorrent: LAN-ONLY at the edge (2026-11-04) -----------
+# Frank's law: these never leave the home boxes. DNS secrecy is NOT a gate —
+# Caddy issues one certificate per name and every one is in public CT logs
+# (crt.sh lists prowlarr/radarr/lidarr/readarr/qbittorrent.franksdank.cloud),
+# so a stranger can send the Host. The gate is the source address: a LAN client
+# proxies through, anything off-LAN gets 404. `remote_ip` reads the real client
+# address because the router DNATs the forward (no proxy hop in between).
 prowlarr.franksdank.cloud {
-	reverse_proxy 192.168.0.112:9696
+	@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
+	handle @lan {
+		reverse_proxy 192.168.0.112:9696
+	}
+	handle {
+		respond 404
+	}
 }
 
 radarr.franksdank.cloud {
-	reverse_proxy 192.168.0.112:7878
+	@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
+	handle @lan {
+		reverse_proxy 192.168.0.112:7878
+	}
+	handle {
+		respond 404
+	}
 }
 
 # --- Sonarr: internal-only by construction (retired 2026-09-04) -----------
@@ -288,15 +338,33 @@ radarr.franksdank.cloud {
 # zero-auth UI to the internet via manual DNS — retired.
 
 lidarr.franksdank.cloud {
-	reverse_proxy 192.168.0.112:8686
+	@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
+	handle @lan {
+		reverse_proxy 192.168.0.112:8686
+	}
+	handle {
+		respond 404
+	}
 }
 
 readarr.franksdank.cloud {
-	reverse_proxy 192.168.0.112:8787
+	@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
+	handle @lan {
+		reverse_proxy 192.168.0.112:8787
+	}
+	handle {
+		respond 404
+	}
 }
 
 qbittorrent.franksdank.cloud {
-	reverse_proxy 192.168.0.112:8080
+	@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
+	handle @lan {
+		reverse_proxy 192.168.0.112:8080
+	}
+	handle {
+		respond 404
+	}
 }
 
 # --- Internal listener: auth decision layer ---------------------------------

dankclaw/pihole/addn-hosts

@@ -26,6 +26,7 @@
 192.168.0.143 romm.franksdank.cloud
 192.168.0.143 files.franksdank.cloud
 192.168.0.143 jelly.franksdank.cloud
+192.168.0.143 audiobooks.franksdank.cloud
 192.168.0.143 radarr.franksdank.cloud
 192.168.0.143 lidarr.franksdank.cloud
 192.168.0.143 readarr.franksdank.cloud

dankserver/.AGENTTRASH/20260916-vllm-radiance/affinity-proxy.py

@@ -1,156 +0,0 @@
-#!/usr/bin/env python3
-"""HTTP front door for the frank-medium single-GPU replicas.
-
-Why two replicas instead of TP=2: one vLLM process spanning both R9700s needs
-RCCL collectives, and RCCL's kernels require PCIe atomics on every
-participating GPU. The second card sits behind chipset bridges advertising
-`AtomicOpsCap: Routing-`, which no software can enable (see the runbook), so
-no collective can launch on it. Two single-GPU replicas use both cards; this
-service keeps the fleet contract of one endpoint on :7770, streaming included.
-
-Routing decisions live in routing.py — spread single-turn requests, pin
-continuations to the replica whose prefix cache holds their context. This file
-only moves bytes: parse enough of the request to route it, forward it verbatim,
-stream the answer back, and report replica health on /health.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import os
-from contextlib import suppress
-from aiohttp import ClientSession, ClientTimeout, web
-
-import routing
-
-PORT = int(os.environ.get("PORT", "7770"))
-HEALTH_TTL = float(os.environ.get("HEALTH_TTL", "10"))
-# Hop-by-hop headers must not be forwarded (RFC 9110 §7.6.1) — forwarding
-# transfer-encoding or content-length of a rewritten body breaks clients.
-HOP_BY_HOP = {
-    "host", "content-length", "transfer-encoding", "connection",
-    "keep-alive", "proxy-authenticate", "proxy-authorization", "te",
-    "trailers", "upgrade",
-}
-
-UPSTREAMS = routing.UPSTREAMS
-# Replica health, refreshed by the watcher. Module state rather than aiohttp app
-# state: the application is frozen once the site is up.
-_UP: list[bool] = [True] * len(UPSTREAMS)
-
-
-async def health(session: ClientSession) -> web.Response:
-    results = await asyncio.gather(
-        *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
-        return_exceptions=True,
-    )
-    lines, up = [], 0
-    for url, res in zip(UPSTREAMS, results):
-        if isinstance(res, Exception) or res.status != 200:
-            lines.append(f"{url} DOWN")
-        else:
-            up += 1
-            lines.append(f"{url} UP")
-    return web.Response(status=200 if up else 503, text="\n".join(lines) + "\n")
-
-
-async def watch_health() -> None:
-    """Refresh the replica up/down table so routing never targets a dead one.
-
-    One HTTP request per replica per interval, in exchange for never handing a
-    client a 502 after a restart.
-    """
-    global _UP
-    session = ClientSession()
-    try:
-        while True:
-            res = await asyncio.gather(
-                *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
-                return_exceptions=True,
-            )
-            _UP = [not isinstance(r, Exception) and r.status == 200 for r in res]
-            await asyncio.sleep(HEALTH_TTL)
-    finally:
-        await session.close()
-
-
-async def proxy(request: web.Request) -> web.StreamResponse:
-    session: ClientSession = request.app["session"]
-    if request.path == "/health":
-        return await health(session)
-
-    body = await request.read()
-    headers = {k: v for k, v in request.headers.items() if k.lower() not in HOP_BY_HOP}
-
-    # Fail over before anything is written to the client. A replica that
-    # restarts is invisible to the health watcher for up to HEALTH_TTL, and a
-    # deploy restarts both replicas — without this retry a 10 s window of
-    # requests would 502 on a service that has a live card the whole time.
-    # vLLM's OpenAI endpoints are stateless per request, so re-sending a
-    # request that never reached an engine is safe; a request that failed
-    # mid-stream is not retried (that path is below).
-    tried: set[int] = set()
-    upstream = None
-    last_error = "no healthy frank-medium replica"
-    while len(tried) < len(UPSTREAMS):
-        idx = routing.route(body, request.remote, _UP)
-        if idx < 0 or idx in tried:
-            break
-        tried.add(idx)
-        target = UPSTREAMS[idx] + request.path_qs
-        try:
-            upstream = await session.request(
-                request.method, target, headers=headers, data=body or None,
-                timeout=ClientTimeout(total=None, sock_connect=10, sock_read=3600),
-            )
-            break
-        except Exception as exc:
-            # Demote at once: the watcher will confirm, but every request until
-            # then should skip this replica rather than pay the same timeout.
-            # Logged because a silent failover hides a card that is down.
-            _UP[idx] = False
-            last_error = f"{target} unreachable: {exc}"
-            print(f"proxy: failover — {target} unreachable ({exc}); retrying "
-                  f"another replica", flush=True)
-    if upstream is None:
-        # 503 when no replica was even reachable through the routing table,
-        # 502 when replicas were tried and all of them failed.
-        status = 502 if tried else 503
-        return web.Response(status=status, text=f"proxy: {last_error}\n")
-
-    out = web.StreamResponse(status=upstream.status, reason=upstream.reason)
-    for k, v in upstream.headers.items():
-        if k.lower() not in HOP_BY_HOP:
-            out.headers[k] = v
-    await out.prepare(request)
-    try:
-        async for chunk in upstream.content.iter_any():
-            await out.write(chunk)
-    except Exception:
-        # The client went away mid-stream (an aborted generation). Nothing to
-        # report: the engine sees its own cancellation.
-        pass
-    finally:
-        upstream.release()
-    with suppress(Exception):
-        await out.write_eof()
-    return out
-
-
-async def main() -> None:
-    app = web.Application(client_max_size=1024 ** 3)
-    app["session"] = ClientSession()
-    app.router.add_route("*", "/{tail:.*}", proxy)
-    runner = web.AppRunner(app)
-    await runner.setup()
-    await web.TCPSite(runner, "0.0.0.0", PORT).start()
-    print(f"proxy: :{PORT} -> {', '.join(UPSTREAMS)} routing={routing.ROUTING} "
-          f"key={routing.AFFINITY_BYTES}B known_max={routing.KNOWN_MAX}", flush=True)
-    watcher = asyncio.create_task(watch_health())
-    with suppress(asyncio.CancelledError):
-        await asyncio.gather(watcher)
-
-
-if __name__ == "__main__":
-    with suppress(KeyboardInterrupt):
-        asyncio.run(main())

dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium-a.container

@@ -1,98 +0,0 @@
-# Quadlet unit: frank-medium replica A (rootless, boot-persistent).
-#
-# Install path on dankserver: ~/.config/containers/systemd/frank-medium-a.container
-# Unit name: frank-medium-a.service (container name: frank-medium-a)
-#
-# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
-# tensor-parallel process across both R9700s (see frank-medium.container, kept
-# as the target). That shape cannot run on this host: RCCL's collective kernels
-# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
-# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
-# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
-# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
-# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
-# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
-# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
-# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
-# atomics. Result: every collective that touches GPU 2 dies at launch.
-# Two independent single-GPU replicas use both cards instead; the proxy unit
-# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
-#
-# SHAPE. vLLM binds 0.0.0.0:7771 on the host network directly; no PublishPort.
-# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
-# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
-# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
-# this shape instead of using the 2x7551-32624 pin, which is only valid at
-# TP=2/262144.
-#
-# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=0 (the first R9700). The
-# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
-# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
-# first and HIP indexes into the filtered list, so a single-valued pair
-# (ROCR=1,HIP=1) leaves an empty device set.
-#
-# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
-# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
-[Unit]
-Description=frank-medium replica A — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #1
-After=network-online.target
-
-[Container]
-ContainerName=frank-medium-a
-Image=localhost/radiance-vllm:0.9.3
-AddDevice=/dev/kfd
-AddDevice=/dev/dri
-GroupAdd=keep-groups
-# The upstream launcher's exact podman posture (rootless).
-PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
-Environment=HIP_VISIBLE_DEVICES=0
-Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
-Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
-Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
-Environment=SERVED_NAMES=frank-medium
-Environment=PORT=7771
-Environment=TP=1
-Environment=MAXSEQS=4
-Environment=CHUNK=8192
-Environment=MAXLEN=114688
-Environment=GPU_UTIL=0.98
-Environment=KV_MEM=0
-Environment=SPEC_METHOD=dflash
-Environment=SPEC=7
-# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
-Volume=/home/dank/models:/models
-# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
-Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
-# Compile cache. Shared with the other replica and with the TP=2 target: the
-# entries are content-addressed, so shapes simply add subdirectories.
-Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
-# The pinned radiance clone: patches applied at every container start.
-Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
-# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
-# shipped r4d.so predates the GDN overflow fix and NaNs this model.
-Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
-# Entrypoint + healthcheck from this checkout, mounted read-only.
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
-# The entrypoint must be the container command; the image's own entrypoint
-# (which the patched script execs at the end) must not run unpatched.
-PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
-# 200 only once the engine serves; during first-start compilation the
-# container is legitimately unhealthy. HealthCmd is split on spaces into an
-# argv array by the podman 5.x v1 generator, hence the file-based probe.
-HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
-HealthInterval=60s
-HealthTimeout=30s
-HealthRetries=3
-HealthStartPeriod=1800s
-Pull=never
-
-[Service]
-# First start compiles Triton/inductor kernels (several minutes) before the
-# engine loads ~20 GiB of weights.
-TimeoutStartSec=1800
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=default.target

dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium-b.container

@@ -1,98 +0,0 @@
-# Quadlet unit: frank-medium replica B (rootless, boot-persistent).
-#
-# Install path on dankserver: ~/.config/containers/systemd/frank-medium-b.container
-# Unit name: frank-medium-b.service (container name: frank-medium-a)
-#
-# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
-# tensor-parallel process across both R9700s (see frank-medium.container, kept
-# as the target). That shape cannot run on this host: RCCL's collective kernels
-# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
-# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
-# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
-# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
-# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
-# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
-# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
-# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
-# atomics. Result: every collective that touches GPU 2 dies at launch.
-# Two independent single-GPU replicas use both cards instead; the proxy unit
-# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
-#
-# SHAPE. vLLM binds 0.0.0.0:7772 on the host network directly; no PublishPort.
-# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
-# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
-# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
-# this shape instead of using the 2x7551-32624 pin, which is only valid at
-# TP=2/262144.
-#
-# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=1 (the second R9700). The
-# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
-# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
-# first and HIP indexes into the filtered list, so a single-valued pair
-# (ROCR=1,HIP=1) leaves an empty device set.
-#
-# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
-# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
-[Unit]
-Description=frank-medium replica B — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #2
-After=network-online.target
-
-[Container]
-ContainerName=frank-medium-b
-Image=localhost/radiance-vllm:0.9.3
-AddDevice=/dev/kfd
-AddDevice=/dev/dri
-GroupAdd=keep-groups
-# The upstream launcher's exact podman posture (rootless).
-PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
-Environment=HIP_VISIBLE_DEVICES=1
-Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
-Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
-Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
-Environment=SERVED_NAMES=frank-medium
-Environment=PORT=7772
-Environment=TP=1
-Environment=MAXSEQS=4
-Environment=CHUNK=8192
-Environment=MAXLEN=114688
-Environment=GPU_UTIL=0.98
-Environment=KV_MEM=0
-Environment=SPEC_METHOD=dflash
-Environment=SPEC=7
-# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
-Volume=/home/dank/models:/models
-# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
-Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
-# Compile cache. Shared with the other replica and with the TP=2 target: the
-# entries are content-addressed, so shapes simply add subdirectories.
-Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
-# The pinned radiance clone: patches applied at every container start.
-Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
-# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
-# shipped r4d.so predates the GDN overflow fix and NaNs this model.
-Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
-# Entrypoint + healthcheck from this checkout, mounted read-only.
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
-# The entrypoint must be the container command; the image's own entrypoint
-# (which the patched script execs at the end) must not run unpatched.
-PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
-# 200 only once the engine serves; during first-start compilation the
-# container is legitimately unhealthy. HealthCmd is split on spaces into an
-# argv array by the podman 5.x v1 generator, hence the file-based probe.
-HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
-HealthInterval=60s
-HealthTimeout=30s
-HealthRetries=3
-HealthStartPeriod=1800s
-Pull=never
-
-[Service]
-# First start compiles Triton/inductor kernels (several minutes) before the
-# engine loads ~20 GiB of weights.
-TimeoutStartSec=1800
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=default.target

dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium-proxy.container

@@ -1,61 +0,0 @@
-# Quadlet unit: frank-medium proxy (rootless, boot-persistent).
-#
-# Install path on dankserver: ~/.config/containers/systemd/frank-medium-proxy.container
-# Unit name: frank-medium-proxy.service (container name: frank-medium-proxy)
-#
-# WHAT IT IS. A prefix-affinity reverse proxy in front of the two single-GPU
-# frank-medium replicas (:7771 on R9700 #1, :7772 on R9700 #2). It preserves
-# the fleet contract: one endpoint on host port 7770 answering as the served
-# model name frank-medium, OpenAI /v1, streaming included.
-#
-# WHY IT EXISTS. TP=2 — one vLLM process spanning both cards — is impossible
-# on this host (see frank-medium-a.container and the runbook: RCCL's kernels
-# need PCIe atomics, and the second card's chipset path advertises
-# AtomicOpsCap: Routing-). Two replicas are how both cards get used.
-#
-# ROUTING. Requests are keyed on a hash of their leading 4 KiB of prompt text,
-# so a conversation family — an agent loop re-sending the same system prompt,
-# tool list and file context every turn — keeps landing on the same replica,
-# where --enable-prefix-caching has that prefix warm. Plain round-robin threw
-# half of those cache hits away. A replica that is down is skipped, and
-# ROUTING=roundrobin restores the old behaviour for A/B measurement. /health
-# answers 200 while at least one replica is up and names the ones that are
-# down. The proxy runs on host networking so it can bind 7770 and reach the
-# replicas on loopback.
-#
-# The script is a few dozen lines of aiohttp, mounted read-only out of
-# /home/dank/serverAppConfigs (deploy.sh pulls that checkout before dispatch).
-# It reuses the same image as the replicas: aiohttp is already there and no
-# extra image has to be pulled on this host.
-[Unit]
-Description=frank-medium proxy — prefix-affinity in front of the two single-GPU replicas
-After=network-online.target
-Wants=frank-medium-a.service frank-medium-b.service
-After=frank-medium-a.service frank-medium-b.service
-
-[Container]
-ContainerName=frank-medium-proxy
-Image=localhost/radiance-vllm:0.9.3
-PodmanArgs=--network=host
-Environment=PORT=7770
-Environment=UPSTREAMS=http://127.0.0.1:7771,http://127.0.0.1:7772
-# The proxy is two files: routing.py holds the replica-selection policy, the
-# server holds the HTTP plumbing. Both are mounted because the server imports
-# the policy by name from its own directory (/usr/local/bin is sys.path[0]).
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/routing.py:/usr/local/bin/routing.py:ro
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/affinity-proxy.py:/usr/local/bin/affinity-proxy.py:ro
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
-PodmanArgs=--entrypoint /usr/local/bin/affinity-proxy.py
-HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
-HealthInterval=60s
-HealthTimeout=30s
-HealthRetries=3
-HealthStartPeriod=60s
-Pull=never
-
-[Service]
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=default.target

dankserver/.AGENTTRASH/20260916-vllm-radiance/routing.py

@@ -1,129 +0,0 @@
-#!/usr/bin/env python3
-"""Routing policy for the frank-medium proxy: which replica serves a request.
-
-Kept apart from the HTTP plumbing in affinity-proxy.py so the decisions can be
-tested without aiohttp, a network or a GPU — the first version of these rules
-shipped with a bug that only appeared on a continuation request, and a test
-that loads this module catches that class of mistake directly.
-
-Two cards, two behaviours, and the workload decides which is right:
-
-* A **single-turn** request is one member of a batch (a benchmark run, a burst
-  of parallel subagents). Spreading those over both replicas lets both cards
-  prefill at once; measured on this host, a 16-request burst of 15k-token
-  prompts finishes in 18.6 s spread versus 30.1 s pinned.
-* A **continuation** re-sends a prefix the replica has already prefilled. Pinned
-  to that replica the turn costs a cache lookup; spread, half the turns pay a
-  cold prefill again (measured: 1096 ms versus 2676 ms mean warm-turn TTFT on
-  90k-token contexts).
-
-ROUTING picks the strategy; the default `adaptive` applies both rules above.
-`affinity` and `roundrobin` exist so the choice can be re-measured, and are not
-production settings.
-"""
-
-from __future__ import annotations
-
-import collections
-import hashlib
-import itertools
-import json
-import os
-
-UPSTREAMS = [u.strip() for u in os.environ.get(
-    "UPSTREAMS", "http://127.0.0.1:7771,http://127.0.0.1:7772").split(",") if u.strip()]
-ROUTING = os.environ.get("ROUTING", "adaptive").strip().lower()
-# Leading bytes of prompt text that identify a conversation's shared prefix.
-# 4 KiB covers a system prompt plus the start of the first user turn, which is
-# where agent loops keep the text they resend every turn.
-AFFINITY_BYTES = int(os.environ.get("AFFINITY_BYTES", "4096"))
-# Prefixes remembered for pinning. Bounded so a long-lived proxy cannot grow
-# without limit; 4096 conversations is far beyond what this fleet runs at once.
-KNOWN_MAX = int(os.environ.get("KNOWN_MAX", "4096"))
-
-_counter = itertools.count()
-# digest -> replica index, insertion-ordered for O(1) LRU eviction. A prefix is
-# remembered as soon as any request carries it; only a continuation is pinned
-# there, because only a continuation can profit from the cache entry.
-_known: "collections.OrderedDict[bytes, int]" = collections.OrderedDict()
-
-
-def affinity_key(body: bytes, remote: str | None) -> tuple[bytes, bool]:
-    """Key a request by its shared prefix, and report whether it continues one.
-
-    Chat and completions bodies both carry their prompt text in JSON, so the
-    leading bytes of that text are the prefix; a body we cannot parse (or an
-    endpoint with no prompt, e.g. /v1/models) falls back to the client address,
-    which still beats nothing for a single-client loop.
-    """
-    fallback = (remote or "anonymous").encode()
-    try:
-        payload = json.loads(body)
-    except Exception:
-        return fallback, False
-
-    text = ""
-    continuation = False
-    if isinstance(payload, dict):
-        if isinstance(payload.get("prompt"), str):
-            text = payload["prompt"]
-        elif isinstance(payload.get("messages"), list):
-            continuation = len(payload["messages"]) > 1
-            parts = []
-            for msg in payload["messages"]:
-                if isinstance(msg, dict):
-                    content = msg.get("content")
-                    if isinstance(content, str):
-                        parts.append(content)
-                    elif isinstance(content, list):  # multimodal content blocks
-                        parts.extend(
-                            blk.get("text", "") for blk in content
-                            if isinstance(blk, dict) and isinstance(blk.get("text"), str)
-                        )
-            text = "\n".join(parts)
-    if not text:
-        return fallback, continuation
-    return text[:AFFINITY_BYTES].encode("utf-8", "replace"), continuation
-
-
-def pick(key: bytes, continuation: bool, up: list[bool]) -> int:
-    """Replica for a request, or -1 when none is available.
-
-    Down replicas are never selected — a rollout must not send traffic to an
-    engine that is still compiling its graphs — and a pin whose replica is down
-    is re-established on a live one rather than kept pointing at a corpse.
-    """
-    digest = hashlib.sha1(key).digest()
-
-    if ROUTING == "roundrobin":
-        start = next(_counter) % len(UPSTREAMS)
-    elif ROUTING == "adaptive":
-        entry = _known.get(digest)
-        if continuation and entry is not None and up[entry]:
-            _known.move_to_end(digest)
-            return entry
-        # Anything that is not a continuation spreads by counter — including a
-        # batch of requests that happen to share one prefix. Hashing the prefix
-        # would send every member of such a batch to whichever card the hash
-        # names (measured: 29.8 s for a 16-request burst, against 18.6 s when
-        # spread). The counter is only advanced by these requests, so batch
-        # members alternate across the cards.
-        start = next(_counter) % len(UPSTREAMS)
-    else:  # affinity
-        start = int.from_bytes(digest[:8], "big") % len(UPSTREAMS)
-
-    for offset in range(len(UPSTREAMS)):
-        idx = (start + offset) % len(UPSTREAMS)
-        if up[idx]:
-            if ROUTING == "adaptive":
-                _known[digest] = idx
-                _known.move_to_end(digest)
-                if len(_known) > KNOWN_MAX:
-                    _known.popitem(last=False)
-            return idx
-    return -1
-
-
-def route(body: bytes, remote: str | None, up: list[bool]) -> int:
-    """Convenience wrapper: decode the request body and choose a replica."""
-    return pick(*affinity_key(body, remote), up)

dankserver/audiobookshelf/quadlet/audiobookshelf.container

@@ -0,0 +1,75 @@
+# Quadlet unit: Audiobookshelf audiobook server (rootless, boot-persistent).
+#
+# Install path on dankserver: ~/.config/containers/systemd/audiobookshelf.container
+# Unit name: audiobookshelf.service (podman 5.x drops the container- prefix)
+#
+# Everything this server writes is its own state — the SQLite library, accounts,
+# playback progress, cached covers — and it lives under
+# /home/dank/Containers/audiobookshelf on the NVMe, so no metadata byte ever
+# lands on the spindle. The audio itself is bound READ-ONLY off the array: the
+# only sanctioned write path into /mnt/entropy is the overlay view
+# (scripts/media-view.sh), Audiobookshelf has no need of it, and a read-only bind
+# makes a stray sidecar write structurally impossible rather than discouraged.
+#
+# The digest is the linux/amd64 manifest of the tag, which is what podman
+# verifies and what RepoDigests records on this single-arch host. Pinning the
+# manifest-list digest instead would let a re-pull resolve to a different image.
+# The upstream version tags stop at 2.19.0 (built 2025-02-02) while `latest` is
+# newer by a year and a half, so `latest@digest` is the only pin that names the
+# build actually running — same reasoning as the jellyfin and romm units.
+
+[Unit]
+Description=Audiobookshelf audiobook server container
+After=network.target
+
+[Container]
+ContainerName=audiobookshelf
+Image=ghcr.io/advplyr/audiobookshelf:latest@sha256:37c2057e150694970442579815c646fe6bdc01c95b9d1547c4c9068be8844ef3
+Network=pasta
+# The names are the image's own: the server reads PORT, CONFIG_PATH, METADATA_PATH
+# and ROUTER_BASE_PATH (measured against the digest below — `podman inspect` shows
+# PORT=80 CONFIG_PATH=/config METADATA_PATH=/metadata in the image ENV, and the
+# server's source greps for exactly these). The ABS_* family printed by the server
+# README is the npm package's naming and is INERT here: with ABS_PORT=8000 the
+# server listened on :80 under base path /audiobookshelf, so the unit sat
+# `active (running)` while every probe to :8000 was refused (measured 2026-11-02,
+# first deploy).
+# ROUTER_BASE_PATH MUST be /audiobookshelf: the SPA in /app/client/dist/index.html is
+# built with `routerBasePath:"/audiobookshelf"` baked in, so it asks for its assets under
+# that prefix no matter what the server is mounted at. With `/` the server answers the
+# page but every script request falls through to the SPA fallback and comes back as
+# text/html — Chrome refuses to execute it and the UI never leaves "Loading…"
+# (measured 2026-11-03; the API and /healthcheck were fine, which is why an API-only
+# verification missed it).
+Environment=PORT=8000
+Environment=ROUTER_BASE_PATH=/audiobookshelf
+Environment=CONFIG_PATH=/config
+Environment=METADATA_PATH=/metadata
+Environment=TZ=America/Vancouver
+PublishPort=8000:8000
+Volume=/home/dank/Containers/audiobookshelf/config:/config
+Volume=/home/dank/Containers/audiobookshelf/metadata:/metadata
+Volume=/mnt/entropy/Audiobooks:/audiobooks:ro
+# The image is Alpine: it ships wget and no curl, so the health check must name
+# the binary that exists or the container is unhealthy while serving perfectly.
+# /healthcheck answers `OK` at the root AND under the base path (measured on both
+# base paths), so the probe below is correct whatever ROUTER_BASE_PATH says.
+HealthCmd=wget -q -O /dev/null http://127.0.0.1:8000/healthcheck
+HealthInterval=30s
+HealthTimeout=10s
+HealthRetries=3
+
+[Service]
+# Refuse to start until the array is a real mount — with the bind source missing,
+# podman invents an empty directory and Audiobookshelf records an empty library as
+# truth. See the header of scripts/wait-for-mount.sh for the blind-boot incident.
+ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /mnt/entropy
+Restart=always
+RestartSec=5
+# The first boot scans the whole audiobook tree. Weighted LOW (default 100) so a
+# scan loses to playback when the array is contended. Proportional share, not a
+# cap: inert while the disk is idle.
+IOWeight=10
+
+[Install]
+WantedBy=default.target

dankserver/compose.yaml

@@ -94,34 +94,12 @@ services:
       timeout: 10s
       retries: 3
 
-  # ═══════════════════════════════════════════════
-  # Jellyfin — media server (movies, TV, music)
-  # ═══════════════════════════════════════════════
-  jellyfin:
-    image: jellyfin/jellyfin:latest
-    container_name: jellyfin
-    restart: unless-stopped
-    ports:
-      - "8096:8096/tcp"
-      - "8920:8920/tcp"
-      - "7359:7359/udp"
-      - "1900:1900/udp"
-    environment:
-      - JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
-      - DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
-    volumes:
-      - /opt/jellyfin/config:/config
-      - /opt/jellyfin/cache:/cache
-      - /mnt/entropy:/data:ro
-    devices:
-      - /dev/dri:/dev/dri
-    group_add:
-      - "992"
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
+  # Jellyfin left this file when it became a Quadlet: the container is owned by
+  # dankserver/jellyfin/quadlet/jellyfin.container, its data roots are under
+  # /home/dank/Containers/jellyfin, and its library arrives through the SSD
+  # overlay at /var/lib/media-meta/view (scripts/media-view.sh). The block that
+  # was here named /opt/jellyfin/* and a read-only array bind — a second owner for
+  # the container name `jellyfin`, wrong on every path. One owner per container.
 
   # ═══════════════════════════════════════════════
   # Calibre-Web — e-book web frontend

dankserver/copyparty/quadlet/copyparty.container

@@ -36,6 +36,9 @@ Volume=/home/dank/Containers/copyparty/cfg:/cfg
 PublishPort=3923:3923
 
 [Service]
+# /w is the array: serving the pre-mount directory instead of the volume shows an
+# empty share and nothing complains, so the mount has to answer first.
+ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /mnt/entropy
 EnvironmentFile=/home/dank/Containers/copyparty/env
 Restart=always
 RestartSec=5

dankserver/host/crypttab

@@ -0,0 +1 @@
+entropy	UUID=cd86c728-459e-4f3a-bcc2-41ecf84685ec	/etc/entropy	nofail,luks

dankserver/host/fstab

@@ -0,0 +1,18 @@
+# /etc/fstab: static file system information.
+#
+# Use 'blkid' to print the universally unique identifier for a
+# device; this may be used with UUID= as a more robust way to name devices
+# that works even if disks are added and removed. See fstab(5).
+#
+# <file system> <mount point>   <type>  <options>       <dump>  <pass>
+# / was on /dev/nvme0n1p2 during curtin installation
+/dev/disk/by-uuid/4f3731d2-fb7b-48e7-a79f-0e33cb8a4162 / ext4 defaults 0 1
+# /boot/efi was on /dev/nvme0n1p1 during curtin installation
+/dev/disk/by-uuid/183C-442D /boot/efi vfat defaults 0 1
+#/dev/disk/by-label/archive-one /mnt/archive-one auto nosuid,nodev,nofail,x-gvfs-show 0 0
+#/dev/disk/by-label/archive-two /mnt/archive-two auto nosuid,nodev,nofail,x-gvfs-show 0 0
+# /dev/md0 is a LUKS container, not a filesystem; its ext4 is mounted via
+# /dev/mapper/entropy on the next line. Mounting /dev/md0 directly fails.
+#/dev/md0 /mnt/md0 ext4 auto,nofail,nosuid,nodev 0 0 
+/dev/mapper/entropy /mnt/entropy ext4 auto,nosuid,nofail,nodev,x-gvfs-show 0 0
+/swapfile none swap sw 0 0

dankserver/host/systemd/media-meta-view.mount

@@ -0,0 +1,30 @@
+# /etc/systemd/system/var-lib-media\x2dmeta-view.mount
+# (repo copy: dankserver/host/systemd/media-meta-view.mount — systemd names a
+# mount unit after its ESCAPED path, so the host file carries the \x2d spelling)
+#
+# Installed by `scripts/media-view.sh install` — the repo copy is the source of
+# truth (fleet law: never edit host state by hand).
+#
+# An OverlayFS merge that shows the media library as one tree while splitting
+# where its bytes live: lowerdir is the RAID6 array (the source of truth, never
+# written), upperdir is a directory on the NVMe that receives every write. The
+# two media servers bind the result at /var/lib/media-meta/view, so Jellyfin's
+# sidecars and artwork are SSD bytes — which is what makes them fast to browse
+# and what keeps Plex's NFO agent off the spinners — while the media itself stays
+# on the array untouched.
+#
+# RequiresMountsFor is load-bearing: without the array mounted, lowerdir is an
+# empty directory and the view would present an empty library as truth.
+
+[Unit]
+Description=SSD-backed overlay view of the media library (array + NVMe metadata layer)
+RequiresMountsFor=/mnt/entropy
+
+[Mount]
+What=overlay
+Where=/var/lib/media-meta/view
+Type=overlay
+Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work
+
+[Install]
+WantedBy=local-fs.target

dankserver/host/systemd/user.slice.d/io-accounting.conf

@@ -0,0 +1,14 @@
+# The io controller is what makes IOWeight= mean anything, and systemd does not
+# enable it below `user.slice` by default: with io accounting off, `io` never
+# appears in `user.slice/cgroup.subtree_control`, so no rootless container can
+# have an `io.weight` file and `IOWeight=` in a quadlet is silently inert.
+# Measured on Dankserver 2026-10-31: setting this property flipped
+# `user.slice/cgroup.subtree_control` from `cpu memory pids` to `cpu io memory
+# pids` immediately, with nothing restarted.
+#
+# This is half the fix; the other half is delegating io to user@1000.service
+# (user@1000.service.d/io-delegate.conf). Both must be in place before a reboot,
+# because the delegation is read when the user manager starts. Law and the
+# verification command: [[Quadlet Lifecycle]].
+[Slice]
+IOAccounting=yes

dankserver/host/systemd/user@1000.service.d/io-delegate.conf

@@ -0,0 +1,12 @@
+# The system manager gives the user manager cpu/memory/pids by default (systemd
+# 259: `systemctl show user@1000.service -p DelegateControllers` → `cpu memory
+# pids`). Without `io` in that set no `io.weight` file can exist anywhere below
+# `user.slice`, so `IOWeight=` in a rootless quadlet is silently inert — the same
+# class of trap as `RequiresMountsFor=`. Granting the controller here is what
+# makes the media servers' I/O share real.
+#
+# Inert until `user@1000.service` starts again, i.e. the next reboot (restarting
+# it would kill every rootless container on the host). Verify after the reboot
+# with the check in [[Quadlet Lifecycle]].
+[Service]
+DelegateControllers=cpu memory pids io

dankserver/jellyfin/quadlet/jellyfin.container

@@ -4,8 +4,9 @@
 # Unit name: jellyfin.service (podman 5.x drops the container- prefix)
 #
 # Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
-# cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
-# anywhere writes to the entropy volume. Every
+# cutover. The library is seen through /var/lib/media-meta/view, the SSD-backed
+# overlay view (scripts/media-view.sh): the array stays read-only and untouched,
+# while the sidecars this server writes land on the NVMe. Every
 # creation-time environment variable of the compose-era container is kept as
 # an Environment= line (zero drift); none of them are secrets, so this unit
 # needs no EnvironmentFile.
@@ -41,7 +42,17 @@ PublishPort=7359:7359/udp
 PublishPort=1900:1900/udp
 Volume=/home/dank/Containers/jellyfin/config:/config
 Volume=/home/dank/Containers/jellyfin/cache:/cache
-Volume=/mnt/entropy:/data:ro
+# The view, writable: Jellyfin is the metadata writer (movie.nfo / tvshow.nfo /
+# episode.nfo + artwork) and those bytes belong on the SSD. The gate in [Service]
+# is the guard: started against a plain, unmounted view directory a scan decides
+# the library is empty and deletes what it cannot see.
+Volume=/var/lib/media-meta/view:/data
+# Big-screen theme for the TV. The repo file replaces the container's "Media
+# Center" theme slot; the upstream dark palette stays live because the file
+# @imports it, so an image bump cannot strand a vendored CSS snapshot. Read-only
+# — the web app only fetches it. A client opts in with
+# Display Settings -> Theme -> "Media Center".
+Volume=/home/dank/serverAppConfigs/dankserver/jellyfin/theme/wmc/theme.css:/jellyfin/jellyfin-web/themes/wmc/theme.css:ro
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 HealthCmd=curl -f http://localhost:8096/health
@@ -50,8 +61,19 @@ HealthTimeout=10s
 HealthRetries=3
 
 [Service]
+# Refuse to start until the bind source is a real mount — see the header of
+# scripts/wait-for-mount.sh for the blind-boot incident this prevents.
+ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /var/lib/media-meta/view /mnt/entropy
 Restart=always
 RestartSec=5
 
+# A library scan reads the array for minutes on end. Weighted LOW (default is
+# 100) so that when the array is contended a scan loses to playback and to every
+# other container this user runs. Proportional share, not a cap: it changes
+# nothing while the disk is idle. 10 is a convention — the requirement is only
+# that it sit far below 100 — and it is inert until user@1000.service is
+# delegated the io controller ([[Quadlet Lifecycle]]).
+IOWeight=10
+
 [Install]
 WantedBy=default.target

dankserver/jellyfin/theme/wmc/theme.css

@@ -0,0 +1,53 @@
+/*
+ * Dank fleet big-screen theme for jellyfin-web.
+ *
+ * Deployed by bind-mounting THIS file over the container's "Media Center"
+ * theme (`/jellyfin/jellyfin-web/themes/wmc/theme.css`) — see the Volume line
+ * in dankserver/jellyfin/quadlet/jellyfin.container. The slot is deliberately
+ * one nobody picks by accident, and its name is the only thing that changes:
+ * the colors are the live upstream dark theme, pulled in by @import below, so
+ * this file never vendors a snapshot of upstream CSS and cannot drift when the
+ * Jellyfin image moves.
+ *
+ * Why it exists: the living-room client (Jellyfin Desktop on danklivingroom)
+ * renders this server-served web app at 3840x2160 with no client-side scaling,
+ * so 14 px body text lands on a big panel viewed from three metres. `zoom` on
+ * the root element is the one lever that scales the whole UI — the layout code
+ * measures layout px, so card grids re-flow to fewer, larger columns instead of
+ * overflowing.
+ *
+ * Adoption is per device, not global: Display Settings -> Theme ->
+ * "Media Center" on the screen that wants it. That is the point — a phone
+ * browser on the same server must not get a 1.75x interface.
+ *
+ * Change the size by editing the single number. 1.5 is mild, 1.75 is the
+ * three-metre default, 2.0 is for a projector.
+ */
+
+@import url("../dark/theme.css");
+
+:root {
+  /* Marker the e2e gate reads back; also a tweak hook for a per-client
+     override injected by anyone who really wants one. */
+  --dank-big-screen: 1;
+  --dank-big-screen-zoom: 1.75;
+}
+
+html {
+  zoom: var(--dank-big-screen-zoom);
+}
+
+/* A remote moves focus in discrete jumps, so the ring has to survive a
+   three-metre read. */
+.emby-focus,
+button:focus-visible,
+a:focus-visible,
+.inputElement:focus-visible {
+  outline: 3px solid var(--accentColor, #00a4dc) !important;
+  outline-offset: 2px !important;
+}
+
+/* Scrollbars are mouse furniture; on a TV they are noise at this size. */
+* {
+  scrollbar-width: none;
+}

dankserver/plex/quadlet/plex.container

@@ -11,12 +11,26 @@ Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
 Volume=/var/lib/plexmediaserver:/config
-Volume=/mnt/entropy:/mnt/entropy:ro
+# The container path stays /mnt/entropy on purpose: every path Plex recorded in
+# its own database is written against it, so swapping the HOST source of this bind
+# to the overlay view re-points Plex at the sidecars without touching its DB.
+# Read-only: Plex consumes the NFO files Jellyfin writes and writes no metadata.
+Volume=/var/lib/media-meta/view:/mnt/entropy:ro
 PublishPort=32400:32400
 
 [Service]
+# Refuse to start until the bind source is a real mount: Plex blind is how paths
+# get marked missing in its own database. Both the view and the array under it
+# must answer — scripts/wait-for-mount.sh names the 2026-10-31 incident.
+ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /var/lib/media-meta/view /mnt/entropy
 Restart=always
 RestartSec=5
 
+# See the jellyfin unit for the reasoning: a section scan gets a LOW weight
+# (default 100) so a walk of 50k files loses the array to playback and to every
+# other container under this user. Inert until user@1000.service holds the io
+# controller ([[Quadlet Lifecycle]]).
+IOWeight=10
+
 [Install]
 WantedBy=default.target

scripts/.AGENTTRASH/20260920-llama-layersplit/bench-spec-ab.py.llamacpp-layersplit

@@ -1,309 +0,0 @@
-#!/usr/bin/env python3
-"""bench-spec-ab.py — settle "which proposer should this endpoint use?" with data.
-
-Runs one payload through several speculative-decoding configurations of the same
-binary, one at a time, with the unit stopped between runs: on a single-GPU host
-there is no room for two copies of a 29 GiB model, and the benchmarking policy
-applies. Every configuration gets identical prompts at temperature 0, and the
-greedy text is compared across them — speculation may buy speed, never change
-the answer.
-
-    bench-spec-ab.py <env-file> <serve-script> [options]
-    bench-spec-ab.py /home/dank/serverAppConfigs/scripts/frank-medium.env \
-                     /home/dank/serverAppConfigs/scripts/serve-frank-medium.sh \
-                     --configs "dflash=draft-dflash@7;ngram=ngram-mod@64;chain=draft-dflash,ngram-mod@7"
-
-Config syntax: `name=SPEC_TYPE[@n_max][|ENV=VAL|ENV=VAL]`, entries joined by `;`.
-The systemd unit is restored in a finally block, so it comes back on whatever the
-env file says even if a configuration crashes the harness.
-
-THREE SHAPES, because a single one answers the wrong question. A proposer can
-only draft text it has seen, so the shape decides the verdict:
-
-  prose       a 1k prompt asking for one sentence. Nothing to copy and nothing
-              repetitive, so only a model-backed proposer can help: this is the
-              shape that compares drafters.
-  echo        a ~1.5k passage of unique lines, "repeat it back verbatim". The
-              output IS the prompt, which is ngram-mod's home turf.
-  structured  "25 JSON records, one per line" — repetitive OUTPUT with no prompt
-              overlap, which is what an ngram matcher is really for in an
-              agentic fleet (config edits, tables, record lists).
-
-echo and structured disable thinking (`chat_template_kwargs`), otherwise the
-model thinks the answer into `reasoning_content`, `content` comes back empty and
-the cross-configuration identity check passes vacuously — which is exactly how
-this harness first "verified" an echo it never looked at.
-
-Results on Dankserver (frank-medium payload, 2026-09-16, temperature 0). Decode:
-
-  shape            dflash=7   ngram-mod(48/64)   chain        (tok/s)
-  prose 1k/512        43.8        17.7 (0 drafts)   43.7
-  echo 1.5k/400       76.8       102.5             200.6
-  structured/400      91.1        17.8 (0 drafts)   91.2
-  identity            70.4        17.8 (0 drafts)   70.4
-
-`draft-mtp,draft-dflash` fails at load (a chain has ONE draft context and MTP
-builds against whatever model that context holds); ngram-mod has no such
-constraint — it is draftless and joins the chain unconditionally.
-"""
-import argparse
-import json
-import os
-import re
-import subprocess
-import sys
-import time
-import urllib.request
-
-FILLER = ("The quick brown fox jumps over the lazy dog near the river bank. "
-          "A cache is a component that stores data so future requests are faster. ")
-TOKENS_PER_REP = 28
-UNIT = "frank-medium.service"
-MODEL = "frank-medium"
-SPEC_GREP = ("adding speculative implementation|- n_max=|- block_size=|creating MTP|"
-             "draft acceptance")
-
-DEFAULT_CONFIGS = [
-    ("none", {"SPEC_TYPE": "none"}),
-    ("mtp-n3", {"SPEC_TYPE": "draft-mtp", "SPEC_DRAFT_N_MAX": "3"}),
-    ("mtp-n7", {"SPEC_TYPE": "draft-mtp", "SPEC_DRAFT_N_MAX": "7"}),
-    ("dflash-n7", {"SPEC_TYPE": "draft-dflash", "SPEC_DRAFT_N_MAX": "7"}),
-]
-
-WORDS = ["latency", "throughput", "cache", "adapter", "scheduler", "allocator",
-         "queue", "kernel", "buffer", "shard", "replica", "window"]
-
-
-def echo_passage(lines=90):
-    """Unique lines: no long span repeats inside the prompt, so an n-gram can
-    only fire when the OUTPUT copies the INPUT."""
-    return "\n".join(
-        "Record %03d: the %s of subsystem %d reports state %d with checksum %05d."
-        % (i, WORDS[i % len(WORDS)], i % 7, i % 3, (i * 7919) % 100000)
-        for i in range(lines))
-
-
-ECHO = (echo_passage() + "\n\nRepeat the passage above verbatim, every line, exactly as "
-        "written, with no commentary before or after it.")
-STRUCTURED = ("Output exactly 25 JSON records, one per line, no prose and no code fence. "
-              'Each line must look like {"id": N, "state": "active", "region": "us-west-2"} '
-              "with N running from 1 to 25.")
-PROSE = FILLER * (1000 // TOKENS_PER_REP) + "\n\nReply with a single short sentence."
-IDENTITY = "List the first five prime numbers, comma separated, then stop."
-NO_THINK = {"chat_template_kwargs": {"enable_thinking": False}}
-
-
-def log(msg, logfile):
-    line = "[" + time.strftime("%H:%M:%S") + "] " + str(msg)
-    print(line, flush=True)
-    with open(logfile, "a") as fh:
-        fh.write(line + "\n")
-
-
-def sh(cmd):
-    return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout.strip()
-
-
-def parse_configs(spec):
-    """`name=SPEC_TYPE[@n_max][|KEY=VALUE]...`, entries joined by ';'.
-
-    KEY=VALUE parts override any env file key, which is how an impl's own window
-    (SPEC_NGRAM_N_MIN/MAX, for instance) gets swept without hand-editing env
-    files."""
-    out = []
-    for chunk in spec.split(";"):
-        chunk = chunk.strip()
-        if not chunk:
-            continue
-        name, _, rest = chunk.partition("=")
-        over = {}
-        if "|" in rest:
-            rest, _, extras = rest.partition("|")
-            for item in extras.split("|"):
-                if "=" in item:
-                    k, _, v = item.partition("=")
-                    over[k.strip()] = v.strip()
-        n_max = None
-        if "@" in rest:
-            rest, _, n_max = rest.partition("@")
-        over["SPEC_TYPE"] = rest.strip()
-        if n_max:
-            over["SPEC_DRAFT_N_MAX"] = n_max.strip()
-        out.append((name.strip(), over))
-    return out
-
-
-def write_env(src, name, overrides):
-    """The env file with this configuration's overrides applied."""
-    path = "/tmp/spec-ab-" + name.replace("/", "_") + ".env"
-    with open(src) as fh:
-        lines = fh.read().splitlines()
-    out = []
-    for line in lines:
-        for k, v in overrides.items():
-            if line.startswith(k + "="):
-                line = k + "=" + v
-                break
-        out.append(line)
-    with open(path, "w") as fh:
-        fh.write("\n".join(out) + "\n")
-    return path
-
-
-def post(base, body, timeout=1800):
-    req = urllib.request.Request(
-        base + "/v1/chat/completions", data=json.dumps(body).encode(),
-        headers={"Content-Type": "application/json"})
-    with urllib.request.urlopen(req, timeout=timeout) as r:
-        return json.load(r)
-
-
-def vram():
-    out = sh("rocm-smi --showmeminfo vram 2>/dev/null | grep 'Used Memory'")
-    return [round(int(x) / 1e9, 2) for x in re.findall(r":\s+(\d+)\s*$", out, re.M)][:2]
-
-
-def wait_health(base, timeout=420):
-    t0 = time.time()
-    while time.time() - t0 < timeout:
-        if not sh("pgrep -x llama-server >/dev/null && echo up"):
-            return False
-        try:
-            with urllib.request.urlopen(base + "/health", timeout=5) as r:
-                if r.status == 200 and json.load(r).get("status") == "ok":
-                    return True
-        except Exception:
-            time.sleep(3)
-    return False
-
-
-def stop_unit():
-    sh("sudo systemctl stop " + UNIT)
-    sh("pkill -x llama-server")
-    for _ in range(40):
-        if not sh("pgrep -x llama-server || true"):
-            return
-        time.sleep(1)
-
-
-def restore_unit(base, logfile):
-    """Bring the endpoint back on the deployed configuration. Runs in a
-    finally, because a harness that can leave production down is a hazard."""
-    log("restoring " + UNIT, logfile)
-    sh("sudo systemctl start " + UNIT)
-    for _ in range(120):
-        try:
-            with urllib.request.urlopen(base + "/health", timeout=5) as r:
-                if json.load(r).get("status") == "ok":
-                    log(UNIT + " healthy again", logfile)
-                    return True
-        except Exception:
-            time.sleep(5)
-    log("WARNING: " + UNIT + " did not come back healthy -- check it by hand", logfile)
-    return False
-
-
-def run_shape(base, prompt, max_tokens, label, extra=None):
-    body = {"model": MODEL, "messages": [{"role": "user", "content": prompt}],
-            "max_tokens": max_tokens, "temperature": 0.0}
-    if extra:
-        body.update(extra)
-    d = post(base, body)
-    tm = d["timings"]
-    msg = d["choices"][0]["message"]
-    content = (msg.get("content") or "").strip()
-    row = {"shape": label,
-           "prompt_n": tm["prompt_n"],
-           "prefill_tps": round(1000 * tm["prompt_n"] / tm["prompt_ms"], 1),
-           "predicted_n": tm["predicted_n"],
-           "decode_tps": round(tm["predicted_per_second"], 2),
-           "draft_n": tm.get("draft_n", 0),
-           "accepted": tm.get("draft_n_accepted", 0),
-           "answer": content,
-           "reasoning_len": len(msg.get("reasoning_content") or "")}
-    if label.startswith("echo"):
-        # How much of the answer really is the passage: a configuration that
-        # "wins" by answering something else is not a win.
-        want = {l.strip() for l in echo_passage().splitlines()}
-        got = [l.strip() for l in content.splitlines() if l.strip()]
-        row["copy_ratio"] = round(sum(1 for l in got if l in want) / len(got), 3) if got else 0.0
-    return row
-
-
-def measure(base):
-    run_shape(base, FILLER * (4000 // TOKENS_PER_REP) +
-              "\n\nReply with a single short sentence.", 32, "warmup")
-    return [
-        run_shape(base, PROSE, 512, "prose 1k/512"),
-        run_shape(base, ECHO, 400, "echo 1.5k/400", NO_THINK),
-        run_shape(base, STRUCTURED, 400, "structured/400", NO_THINK),
-        run_shape(base, IDENTITY, 192, "identity"),
-    ]
-
-
-def main():
-    ap = argparse.ArgumentParser()
-    ap.add_argument("env_file")
-    ap.add_argument("serve_script")
-    ap.add_argument("--base-url", default="http://127.0.0.1:7770")
-    ap.add_argument("--log", default="/tmp/bench-spec-ab.log")
-    ap.add_argument("--configs", default=None,
-                    help="name=SPEC_TYPE[@n_max][|ENV=VAL] joined by ';' "
-                         "(default: the four configs measured on 2026-09-16)")
-    ap.add_argument("--json", default="/tmp/bench-spec-ab.json")
-    args = ap.parse_args()
-
-    configs = parse_configs(args.configs) if args.configs else DEFAULT_CONFIGS
-    # Absolute paths: the launcher must not depend on the caller's cwd, and a
-    # relative path plus a `cd` to its own directory double-counts the prefix.
-    env_file = os.path.abspath(args.env_file)
-    serve_script = os.path.abspath(args.serve_script)
-    base, logfile = args.base_url, args.log
-    results = {}
-    log("spec A/B start: " + ", ".join(n for n, _ in configs) +
-        " (unit " + UNIT + " stopped for the duration)", logfile)
-    try:
-        stop_unit()
-        for name, over in configs:
-            env_path = write_env(env_file, name, over)
-            logf = "/tmp/spec-ab-" + name.replace("/", "_") + ".log"
-            sh("FRANK_MEDIUM_ENV_FILE=" + env_path + " setsid bash " + serve_script +
-               " > " + logf + " 2>&1 < /dev/null &")
-            if not wait_health(base):
-                log(name + ": FAILED to become healthy -- " + sh("tail -3 " + logf), logfile)
-                results[name] = {"error": "not healthy", "tail": sh("tail -6 " + logf)}
-                stop_unit()
-                continue
-            rows = measure(base)
-            results[name] = {"vram_gb": vram(), "rows": rows,
-                             "spec_log": sh("grep -E '" + SPEC_GREP + "' " + logf + " | tail -4")}
-            for r in rows:
-                log(name + " | " + r["shape"] + " prefill " + str(r["prefill_tps"]) +
-                    " tok/s | decode " + str(r["decode_tps"]) + " tok/s | draft " +
-                    str(r["draft_n"]) + "/" + str(r["accepted"]) +
-                    (" (" + str(round(100.0 * r["accepted"] / r["draft_n"])) + " %)"
-                     if r["draft_n"] else "") +
-                    (" | copy " + str(r["copy_ratio"]) if "copy_ratio" in r else "") +
-                    (" | EMPTY ANSWER" if not r["answer"] else ""), logfile)
-            log(name + " | vram " + str(results[name]["vram_gb"]), logfile)
-            stop_unit()
-    finally:
-        restore_unit(base, logfile)
-
-    with open(args.json, "w") as fh:
-        json.dump(results, fh, indent=2)
-    for shape in ("prose 1k/512", "echo 1.5k/400", "structured/400", "identity"):
-        texts = [r["answer"] for v in results.values() if "rows" in v
-                 for r in v["rows"] if r["shape"] == shape]
-        if any(not t for t in texts):
-            log("WARNING: '" + shape + "' produced an empty answer in some configuration -- "
-                "its identity check proves nothing", logfile)
-        elif len(set(texts)) > 1:
-            log("WARNING: configurations disagree on greedy text for '" + shape +
-                "' -- speculation is changing answers", logfile)
-    log("done; results in " + args.json, logfile)
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())

scripts/.AGENTTRASH/20260920-llama-layersplit/frank-medium.env.llamacpp-layersplit

@@ -1,318 +0,0 @@
-# Tunables for frank-medium (orcarouter/Qwen3.8-27B-Uncensored-GGUF Q8_0 +
-# z-lab/Qwen3.8-27B-DFlash2-GGUF Q4_K_M drafter), mainline llama.cpp on
-# Dankserver, 2x AMD Radeon AI PRO R9700 (gfx1201).  Since 2026-09-16 this is
-# the SOLE LLM runtime on the box: the vllm-radiance replica pair and its
-# routing proxy are retired.
-#
-# Hardware: two R9700 (Navi 48, RDNA4, 32,624 MiB each), no XGMI, different
-# IOMMU groups -- every cross-card byte goes over PCIe.  The Raphael iGPU
-# (gfx1036/Radeon 610M) enumerates as HIP device 2 and is hidden below so
-# nothing can drift onto shared system RAM.
-#
-# WHY LAYER SPLIT AND NOT TENSOR (operator call, 2026-09-16): tensor parallel
-# needs a cross-card allreduce per layer, and this host cannot dispatch one.
-# RCCL's kernels carry a hidden hostcall-buffer parameter, and ROCr refuses it
-# on a device without PCIe atomics -- the second card sits behind the B650
-# chipset bridges (AtomicOpsCap: Routing-, read-only), so every collective that
-# touches it dies with hipErrorIllegalState.  Measured on this box 2026-09-15
-# (llama-bench, pp4096 / tg128, f16 KV, fa on):
-#
-#   single R9700    1160.9 / 19.42
-#   layer  2x       1991.2 / 18.76      (859.1 / 17.64 at depth 32768)
-#   tensor 2x        271.8 / 26.26      (no RCCL: butterfly fallback, -7x prefill)
-#
-# Layer split distributes layers in order; only the activation at the boundary
-# crosses the wire, once per forward pass.  That is the "pipeline parallelism"
-# this service wanted, and it needs no collective at all.
-#
-# Runtime: mainline llama.cpp master fb27a525 (2026-09-16), gfx1201, built by
-# scripts/build-llama-rdna4.sh against a private ROCm 7.14.0a20260624 toolchain
-# that is bundled next to the binary (RPATH $ORIGIN/../lib, no LD_LIBRARY_PATH).
-# The ref is pinned by TWO features: DFlash2 (ggml-org/llama.cpp#27342 -- the
-# drafter is detected from its GGUF) and qwen35 MTP.  Do not float it.
-#
-# Model: orcarouter/Qwen3.8-27B-Uncensored-GGUF @ Q8_0, 27.0 GiB, arch qwen35
-# (verified from the GGUF header 2026-09-19: 262144 trained window, 24 heads /
-# 4 KV heads, nextn_predict_layers=1).  The file ALSO carries the model's MTP
-# head (blk.64.nextn.*), so --spec-type draft-mtp is available without a second
-# copy in VRAM -- one SPEC_TYPE change, the drafter below stays on disk.
-#
-# Drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF @ Q4_K_M, 1.06 GiB (operator call
-# 2026-09-19: the Q4 rung -- the drafter's job is throughput, not output
-# quality, and the target verifies every drafted token).  5 layers, hidden
-# 5120, a block-diffusion proposal (block_size 8) under a 2048-token sliding
-# window (dflash.attention.sliding_window=2048, verified from the header).
-# The window is what makes it cheap HERE: its KV is bounded at 2048 positions
-# (~40 MiB) instead of spanning the context (see SIZING).
-#
-# Architecture (GGUF arch qwen35): 64 layers as 16 x (3 x (Gated DeltaNet ->
-# FFN) + 1 x (Gated Attention -> FFN)) -- 48 linear-attention layers whose
-# per-sequence state is CONSTANT-SIZE, 16 full-attention layers at 4 KV heads x
-# 256 head_dim.  A token therefore costs 2 x 4 x 256 x 2 B = 64 KiB of KV at f16
-# in the attention layers only, plus ~0.16 GiB per sequence of f32 DeltaNet
-# state at any depth.
-#
-# SIZING, the deployed shape (32:33 / ctx 327680 / 6144x512 / two slots /
-# DFlash2 Q4_K_M), measured 2026-09-19 after the model swap (rocm-smi):
-#
-#   card 0 (x16)  26.9 GiB used, 5.7 GiB free
-#   card 1 (x1)   27.8 GiB used, 4.8 GiB free
-#
-# The predecessor's measured table (rocm-smi, 2026-09-16, UD-Q8_K_XL target)
-# stays as the calibration history:
-#
-#   CTX     split  batch  slots  drafter  GPU 0 free  GPU 1 free   note (ubatch 1536)
-#   262144   1,1   2048   1      DFlash2   8.64 GiB    4.43 GiB   day's start
-#   262144  39,26  6144   2      DFlash2   2.42 GiB    7.20 GiB
-#   262144  38,27  6144   2      DFlash2   3.85 GiB    5.77 GiB
-#   262144  37,28  6144   2      DFlash2   4.28 GiB    5.34 GiB
-#   262144  36,29  6144   2      DFlash2   4.71 GiB    4.91 GiB
-#   262144  35,30  6144   2      DFlash2   5.14 GiB    4.48 GiB
-#   262144  40,25  6144   2      DSpark    1.87 GiB    1.86 GiB   (retired)
-#
-# ...and the same deployed shape re-measured at ubatch 512, after llama-bench
-# showed 512 is the throughput optimum and costs 3.71 GiB less in total (see the
-# batch block below):
-#
-#   262144  36,29  6144   512    2      DFlash2   6.01 GiB    7.32 GiB   ubatch only
-#   262144  35,30  6144   512    2      DFlash2   6.44 GiB    6.90 GiB
-#
-# ONE -ts STEP IS ONE LAYER, AND LAYERS ARE NOT THE SAME SIZE.  This is a hybrid
-# stack -- 16 x (3 Gated DeltaNet + 1 Gated Attention) blocks -- so moving a
-# DeltaNet layer shifts ~0.43 GiB while an attention layer shifts ~1.43 GiB
-# (0.45 of weights plus ~1.0 GiB of its own f16 KV at 262144).  Both step sizes
-# were measured, which is why the balance is found by measurement rather than by
-# interpolating between splits.
-#
-# WHY THE SPLIT IS 32:33 (operator call, 2026-09-19).  The operator asked for
-# the tensors BALANCED across the two cards, which for 65 layers is 32:33 --
-# the most even split that exists.  The predecessor's 35:30 skew existed to lean
-# work onto the x16 card (ROCm0), because ROCm1's Gen4 x1 chipset path made it
-# the 98 % utilization bottleneck; the measured trade of the balanced point is
-# that the x1 card runs hotter while the x16 card idles a little more, and the
-# operator accepted that for symmetry.  Do not re-skew without asking.
-#
-# THE TWO CARDS' PATHS ARE NOT SYMMETRIC (measured, 2026-09-17): ROCm0
-# (03:00.0) hangs off CPU root port 00:01.1 at Gen4 x16 end to end (~31.5 GB/s);
-# ROCm1 (0b:00.0) rides the B650 chipset through bridges that each negotiate
-# Gen4 x1 (~1.97 GB/s).  The layer-split boundary itself is not the cost
-# (~5.2 MB of activation per 512-token ubatch rides the wire); the x1 card is
-# COMPUTE-bound, not PCIe-bound.
-#
-# A FAULT TO WATCH.  A 39:26 split, at 1x batch / 1 slot, died on its eighth
-# request: the amdgpu driver logged SDMA0 UTCL2 page faults and the engine
-# surfaced `ROCm error: an illegal memory access was encountered` -> ggml_abort,
-# i.e. a fault in the cross-card boundary copy, this layer split's hot path.  It
-# has not reproduced since, but the signature is worth recognising -- it is NOT
-# an OOM, and no amount of headroom prevents it.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
-# those bare vars; 2026-08-13 incident).  llama.cpp takes --host/--port/--temp
-# as flags only.
-
-# llama-server: mainline fb27a525 built for gfx1201 by scripts/build-llama-rdna4.sh.
-LLAMA_BIN=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server
-
-# Payload, fetched with scripts/download-frank-medium-gguf.sh.
-MODEL_DIR=/home/dank/models/Qwen3.8-27B-Uncensored-GGUF
-MODEL_FILE=Qwen3.8-27B-Uncensored-Q8_0.gguf
-# The f16 vision projector, same repo as the target.  llama-server loads it via
-# the explicit --mmproj flag (no autoload in this build); the serve script
-# refuses to start without it when vision is enabled, so a missing file is a
-# deploy error, not a text-only server that pretends to be multimodal.
-MM_PROJ_FILE=mmproj-Qwen3.8-27B-Uncensored-f16.gguf
-DRAFT_MODEL_DIR=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF
-DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q4_K_M.gguf
-
-# Fleet identity.  7770 is the fleet's dankserver endpoint; the alias is the id
-# at /v1/models and what every client asks for.
-ALIAS=frank-medium
-BIND_ADDR=0.0.0.0
-LISTEN_PORT=7770
-# Device placement.  HIP_VISIBLE_DEVICES hides the Raphael iGPU (HIP device 2)
-# so nothing can drift onto shared system RAM; DEVICES is what llama.cpp splits
-# across, and neither of the two cards has an XGMI link to the other.
-HIP_VISIBLE_DEVICES=0,1
-DEVICES=ROCm0,ROCm1
-SPLIT_MODE=layer
-
-# 32:33 -- balanced (operator call 2026-09-19): 65 layers split as evenly as
-# they can be, ROCm1 (the last-layers card, which also carries the drafter and
-# the output layers) holding the extra one.  See the WHY THE SPLIT IS 32:33
-# block above for the trade against the predecessor's 35:30 x16-lean.
-TENSOR_SPLIT=32,33
-
-NGL=999
-# Auto-fit has no business resizing a context we set explicitly.
-FIT=off
-FLASH_ATTN=on
-
-# 327680 positions over TWO slots, so each conversation gets 163840 --
-# --parallel DIVIDES --ctx-size, so more slots would shrink every
-# conversation rather than grow the cache.  163840 is inside the checkpoint's
-# trained 262144 window, so the whole pool is trained territory (no rope
-# extrapolation, unlike the predecessor's 393216).
-CTX_SIZE=327680
-PARALLEL=2
-
-# MEASURED 2026-09-16 with scripts/bench-batch-ubatch.sh on this box: llama-bench,
-# a 64k prefill plus a 128-token decode with 64k of KV already resident, 3 reps,
-# one invocation per cell, the deployed split and f16 KV.  --ubatch-size is the
-# physical batch the graph is built at -- and the only one of the two knobs that
-# costs VRAM:
-#
-#   batch  ubatch   prefill tok/s        GPU 0 / GPU 1 free
-#   6144   1536    1164.96 +- 2.30      10.28 / 11.71 GiB
-#   6144    768    1135.03 +- 1.09      11.29 / 13.28 GiB
-#   6144    512    1192.93 +- 0.91      11.63 / 13.81 GiB   <- deployed
-#   6144    384    1068.04 +- 0.79      11.80 / 14.07 GiB
-#   6144    256    1045.98 +- 1.69      11.97 / 14.33 GiB
-#   4096   1536    1161.90 +- 0.96      10.28 / 11.71 GiB
-#   2048   1536    1170.23 +- 0.86      10.28 / 11.71 GiB
-#   2048    512    1193.88 +- 1.01      11.63 / 13.81 GiB
-#
-# THE PLATEAU STARTS AT 512.  512 -> 1536 buys ~2 % of prefill -- inside the
-# run-to-run spread, and the 512 cells were the faster pair in BOTH batch
-# columns -- while costing 1.35 GiB on ROCm0 and 2.10 GiB on ROCm1, so the
-# deployment takes the cheap end.  BELOW 512 the SMs are no longer saturated:
-# 384 and 256 fall ~11 % and ~12 %.  The logical batch is free either way --
-# 2048, 4096 and 6144 land within 2 % of each other at a fixed ubatch and use
-# the SAME VRAM, because the compute buffers are sized by ubatch alone.
-#
-# The prefill figure is a property of the SHAPE, not of this knob: the same
-# sweep with no depth in front of it (depth 0) runs 1543 tok/s at ubatch 256 and
-# 1723 at 1536 -- the plateau is in the same place, and the ~32 % between the two
-# tables is what attending over 64k of resident KV costs.  Decode is 15.88 tok/s
-# at depth 65536 on EVERY ubatch (17.80 at depth 0), so this knob cannot touch
-# generation at all.
-#
-# 8192/2048 stays untested under this drafter (the "2020 MiB pp buffer fails on
-# ROCm1" note was measured under DSpark, when it had 0.08 GiB free), and there is
-# no longer a reason to want it: batch buys nothing measurable here.
-BATCH_SIZE=6144
-UBATCH_SIZE=512
-
-# Threads cover tokenization, sampling and HTTP only: every layer is on a GPU.
-NUM_THREADS=8
-THREAD_BATCH=8
-
-# f16 K/V: an agentic checkpoint is the wrong place to spend KV precision.
-CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
-# Context checkpoints: 0 -- no per-slot KV snapshots at all.  The build's
-# default is 32 (this binary's --help), so the 0 below is a deliberate
-# deviation, not an inherited default: a checkpoint is a VRAM->host-RAM copy
-# of part of a slot's KV, taken so a partially-matching prompt can be restored
-# instead of re-prefilled.  This endpoint trades that away -- its reuse comes
-# from the slot prompt and from the RAM-side prompt cache below.  Stated
-# explicitly so the tunable stays visible rather than merely absent.
-CTX_CHECKPOINTS=0
-# PROMPT CACHE, HOST RAM -- this is the one cache tunable that is NOT VRAM.
-# --cache-ram caps the total bytes of saved prompt states
-# (server_prompt_cache, tools/server/server-task.cpp): update() evicts
-# oldest-first past the cap, and alloc() REFUSES outright any single state
-# bigger than the whole cap.  A state is the serialized KV of a whole sequence
-# plus its DeltaNet state plus the draft context's own KV, so at this shape it
-# costs ~68 KiB per token --
-#   16 full-attention layers x 4 kv heads x 256 dim x 2 B x 2 (K,V) = 64 KiB
-#   plus the draft context's window-bounded KV (a ~40 MiB constant for DFlash2)
-# -- which makes the cap a TOKEN budget in disguise: 16384 MiB holds ~240k
-# tokens, i.e. a FULL 163840-token slot (~10.6 GiB) with ~5 GiB of the budget
-# spare for a second entry.
-#
-# The cost is host RAM, allocated on demand and never pinned (plain heap
-# vectors), so no VRAM number above moves.  dankserver has 62 GiB beside the
-# media-stack co-tenants, so the cache is bounded rather than unlimited (-1):
-# the seam test refuses a budget over half the host's RAM.
-CACHE_RAM=16384
-
-# Sampling defaults for requests that send none -- the values the vLLM
-# predecessor served on this endpoint (`--override-generation-config`), so a
-# client sees no change across the runtime swap.  The model card's own
-# recommendation is temp 1.0 / top_p 0.95; the harness sets its own per model in
-# models.yml, so this is the floor, not the contract.
-TEMPERATURE=0.7
-TOP_P=0.95
-TOP_K=20
-MIN_P=0.0
-PRESENCE_PENALTY=0.0
-
-# Reasoning on, projected into reasoning_content (deepseek format) with no
-# budget: the model decides how long to think.
-REASONING=on
-REASONING_FORMAT=deepseek
-REASONING_BUDGET=-1
-
-# Chat template: empty = the one embedded in the GGUF (Qwen3.8's own).
-CHAT_TEMPLATE_FILE=
-CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
-
-# Speculative decoding: the DFlash2 sidecar (operator call 2026-09-19).  The
-# target file also carries its own MTP head (nextn_predict_layers=1), so
-# draft-mtp is one SPEC_TYPE change away -- the drafter file stays on disk.
-# n_max 7 is the measured DFlash2 winner (see the table below); the drafter
-# rides ROCm1, the card that holds the last layers.
-SPEC_TYPE=draft-dflash
-SPEC_DRAFT_N_MAX=7
-SPEC_DRAFT_NGL=999
-SPEC_DRAFT_DEVICE=ROCm1
-SPEC_DRAFT_TYPE_K=f16
-SPEC_DRAFT_TYPE_V=f16
-
-# DFlash2 vs the model's own MTP head -- MEASURED 2026-09-16 on the predecessor
-# payload (temperature 0, 1k prompt / 512 out; the autoregressive baseline and
-# both MTP depths were run in one session, same binary, same prompts):
-#
-#   none            17.7 tok/s                     card1 26.07 GB
-#   draft-mtp  n=3  32.4 tok/s  58 % accepted       card1 29.35 GB
-#   draft-mtp  n=7  26.7 tok/s  33 % accepted       card1 29.65 GB
-#   draft-dflash 7  43.5 tok/s  41 % accepted       card1 29.42 GB
-#
-# All four produced IDENTICAL greedy text, so verification is lossless
-# whichever proposer runs.  DFlash2 is the faster proposer (2.46x over
-# autoregressive, 34 % over the best MTP).  The MTP head adds ~3.3 GB to the
-# card that holds it, against 1.06 GB for the Q4_K_M drafter's file -- at the
-# 327680 pool the drafter's window-bounded KV (~40 MiB) leaves headroom on
-# both cards, so the speed wins.
-#
-# THE TWO CANNOT BE CHAINED. `--spec-type draft-mtp,draft-dflash` dies at load:
-#   context type MTP requested but model doesn't contain MTP layers
-#   failed to create MTP context
-# A chain has ONE draft context, and MTP builds against whatever model that
-# context holds -- here the DFlash2 file, which has no MTP layers. MTP's head is
-# only loaded when MTP IS the draft model ("creating MTP draft context against
-# the target model ..."), so a chain of the two is a structural dead end, not a
-# tuning question. See serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md.
-
-# ngram-mod: the draftless n-gram matcher (CPU hash table in host RAM, no VRAM,
-# no model file, no draft context).  It joins the chain unconditionally --
-# unlike MTP, which needs a draft context and therefore cannot share one with
-# DFlash2.  NOT CURRENTLY ACTIVE: SPEC_TYPE is draft-dflash alone, and the serve
-# script only passes --spec-ngram-mod-* when the type contains "ngram".  The
-# window below is retained because the DFlash2 + ngram chain was measured free
-# on prose and 2.6x on prompt-echo -- one SPEC_TYPE change brings it back.
-#
-# WINDOW MEASURED 2026-09-16 (chain at each window, decode tok/s):
-#
-#   n_min/n_max    prose 1k/512   prompt-echo 1.5k/400
-#   48/64          43.9           200.6
-#   16/32          36.0           161.0
-#    8/16          36.2           113.8
-#
-# The long window is the right one, and not only because it wins on echo: a
-# SHORT window makes the matcher propose on short fragments of ordinary prose,
-# where the proposals are rejected and the verification still costs -- which is
-# the "ngram hurts" result the fleet measured on other payloads twice before.
-# With 48/64 the chain is byte-identical to DFlash2 alone on prose (same 119
-# drafts / 49 accepted) and on the identity prompt (70/53).
-#
-# Do not shorten these without re-running scripts/bench-spec-ab.py: the
-# no-penalty property is what makes the chain safe as a default.
-SPEC_NGRAM_N_MATCH=24
-SPEC_NGRAM_N_MIN=48
-SPEC_NGRAM_N_MAX=64
-
-# Vision enabled: llama-server loads mmproj-Qwen3.8-27B-Uncensored-f16.gguf
-# (931 MiB, lands on ROCm0 -- the fit charges it there) via the explicit
-# --mmproj flag.  This keeps frank-medium multimodal across the model swap.
-NO_MMPROJ=0

scripts/.AGENTTRASH/20260920-llama-layersplit/frank-medium.service.llamacpp-layersplit

@@ -1,20 +0,0 @@
-[Unit]
-Description=llama.cpp Qwen3.8-27B Uncensored UD-Q8_K_XL + DFlash2 drafter (dankserver, frank-medium, 2x Radeon AI PRO R9700)
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-# The engine pins staging buffers on the first card; keep the memlock budget
-# unbounded so an RLIMIT that is smaller than the split buffers cannot fail the
-# load.  Parity with the other LLM units on this fleet.
-LimitMEMLOCK=infinity
-Environment="FRANK_MEDIUM_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-medium.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-medium.sh
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

scripts/.AGENTTRASH/20260920-llama-layersplit/serve-frank-medium.sh.llamacpp-layersplit

@@ -1,203 +0,0 @@
-#!/usr/bin/env bash
-# Serve ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF @ UD-Q8_K_XL as
-# frank-medium on Dankserver's two Radeon AI PRO R9700 (gfx1201), with the
-# z-lab DFlash2 block-diffusion drafter, using mainline llama.cpp.
-#
-# Runtime: llama.cpp master fb27a525 (2026-09-16), a gfx1201 HIP build that
-# bundles its own ROCm 7.14 runtime (RPATH $ORIGIN/../lib), produced by
-# scripts/build-llama-rdna4.sh.  Two features pin that ref: DFlash2
-# (ggml-org/llama.cpp#27342) and qwen35 MTP.  The charlie12345/ROCmFPX fork
-# this deployment replaced has neither (its tip is c49ebdb, 2026-08-22).
-#
-# Parallelism: --split-mode layer across both cards.  Tensor split would need a
-# cross-card allreduce and this host cannot dispatch one -- RCCL's kernels
-# require a hostcall buffer that ROCr refuses on a card without PCIe atomics,
-# and the second R9700 sits behind the chipset (AtomicOpsCap: Routing-).  Layer
-# split crosses the wire once per forward pass instead of once per layer.
-#
-# The vLLM replica pair and its routing proxy were retired with this cutover:
-# one process, one model, both cards.
-#
-# Tunables come from frank-medium.env unless FRANK_MEDIUM_ENV_FILE overrides it.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_MEDIUM_ENV_FILE:-$SCRIPT_DIR/frank-medium.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) ------------------
-: "${LLAMA_BIN:=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF}"
-: "${MODEL_FILE:=Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
-: "${DRAFT_MODEL_DIR:=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF}"
-: "${DRAFT_MODEL_FILE:=Qwen3.8-27B-DFlash2-Q8_0.gguf}"
-: "${ALIAS:=frank-medium}"
-: "${BIND_ADDR:=0.0.0.0}"
-: "${LISTEN_PORT:=7770}"
-
-# Devices.  HIP_VISIBLE_DEVICES hides the Raphael iGPU (HIP device 2) so nothing
-# can drift onto shared system RAM; DEVICES is what llama.cpp splits across.
-: "${HIP_VISIBLE_DEVICES:=0,1}"
-: "${DEVICES:=ROCm0,ROCm1}"
-: "${SPLIT_MODE:=layer}"
-: "${TENSOR_SPLIT:=1,1}"
-: "${NGL:=999}"
-: "${FIT:=off}"
-: "${FLASH_ATTN:=on}"
-
-: "${CTX_SIZE:=131072}"
-: "${PARALLEL:=1}"
-: "${BATCH_SIZE:=2048}"
-: "${UBATCH_SIZE:=512}"
-: "${NUM_THREADS:=8}"
-: "${THREAD_BATCH:=8}"
-: "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-: "${CTX_CHECKPOINTS:=8}"
-: "${CACHE_RAM:=8192}"
-
-# Sampling defaults for silent clients: the values the vLLM predecessor served.
-: "${TEMPERATURE:=0.7}"
-: "${TOP_P:=0.95}"
-: "${TOP_K:=20}"
-: "${MIN_P:=0.0}"
-: "${PRESENCE_PENALTY:=0.0}"
-
-: "${REASONING:=on}"
-: "${REASONING_FORMAT:=deepseek}"
-: "${REASONING_BUDGET:=-1}"
-: "${CHAT_TEMPLATE_FILE:=}"
-: "${CHAT_TEMPLATE_KWARGS:={\"enable_thinking\":true,\"preserve_thinking\":true}}"
-
-# Speculative decoding: draft-dflash or draft-dspark (both are MODEL drafters --
-# a file, not a flag), or draft-mtp (the target GGUF's own MTP head, no draft
-# file).  The drafter's own tuning knobs (n-min, p-min, p-split, poll) are left
-# at the build's defaults: they were only ever passed for dspark, and dspark is
-# not the deployed drafter.
-: "${SPEC_TYPE:=draft-dflash}"
-: "${SPEC_DRAFT_N_MAX:=7}"
-: "${SPEC_DRAFT_NGL:=999}"
-: "${SPEC_DRAFT_DEVICE:=}"
-: "${SPEC_DRAFT_TYPE_K:=f16}"
-: "${SPEC_DRAFT_TYPE_V:=f16}"
-: "${SPEC_NGRAM_N_MATCH:=24}"
-: "${SPEC_NGRAM_N_MIN:=48}"
-: "${SPEC_NGRAM_N_MAX:=64}"
-
-NO_MMPROJ="${NO_MMPROJ:-0}"
-: "${MM_PROJ_FILE:=mmproj-BF16.gguf}"
-
-export HIP_VISIBLE_DEVICES
-
-MODEL="$MODEL_DIR/$MODEL_FILE"
-DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
-
-if [[ ! -x "$LLAMA_BIN" ]]; then
-    echo "missing executable runtime: $LLAMA_BIN" >&2
-    echo "build it with: bash $SCRIPT_DIR/build-llama-rdna4.sh" >&2
-    exit 1
-fi
-if [[ ! -r "$MODEL" ]]; then
-    echo "missing or unreadable target model: $MODEL" >&2
-    echo "fetch it with: bash $SCRIPT_DIR/download-frank-medium-gguf.sh" >&2
-    exit 1
-fi
-if [[ "$NO_MMPROJ" != "1" ]]; then
-    MM_PROJ="$MODEL_DIR/$MM_PROJ_FILE"
-    if [[ ! -r "$MM_PROJ" ]]; then
-        echo "vision is enabled but the projector is missing: $MM_PROJ" >&2
-        echo "fetch it with: bash $SCRIPT_DIR/download-frank-medium-gguf.sh" >&2
-        exit 1
-    fi
-fi
-if [[ -n "$CHAT_TEMPLATE_FILE" && ! -r "$CHAT_TEMPLATE_FILE" ]]; then
-    echo "missing or unreadable chat template: $CHAT_TEMPLATE_FILE" >&2
-    exit 1
-fi
-
-# --- speculative decoding -------------------------------------------------
-SPEC_ARGS=()
-if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
-    SPEC_ARGS=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
-    if [[ "$SPEC_TYPE" == *dflash* || "$SPEC_TYPE" == *dspark* ]]; then
-        if [[ ! -r "$DRAFT_MODEL" ]]; then
-            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
-            exit 1
-        fi
-        SPEC_ARGS+=(
-            --spec-draft-model "$DRAFT_MODEL"
-            --spec-draft-ngl "$SPEC_DRAFT_NGL"
-            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
-            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
-        )
-    fi
-    # ngram-mod is draftless: it needs no draft context and its window is its
-    # own (--spec-ngram-mod-*), so it can ride in the chain beside a model
-    # drafter. It also costs no VRAM -- a CPU hash table.
-    if [[ "$SPEC_TYPE" == *ngram* ]]; then
-        SPEC_ARGS+=(
-            --spec-ngram-mod-n-match "$SPEC_NGRAM_N_MATCH"
-            --spec-ngram-mod-n-min "$SPEC_NGRAM_N_MIN"
-            --spec-ngram-mod-n-max "$SPEC_NGRAM_N_MAX"
-        )
-    fi
-    if [[ -n "$SPEC_DRAFT_DEVICE" ]]; then
-        SPEC_ARGS+=(--spec-draft-device "$SPEC_DRAFT_DEVICE")
-    fi
-fi
-
-# --- template + placement -------------------------------------------------
-TEMPLATE_ARGS=()
-if [[ -n "$CHAT_TEMPLATE_FILE" ]]; then
-    TEMPLATE_ARGS+=(--chat-template-file "$CHAT_TEMPLATE_FILE")
-fi
-TEMPLATE_ARGS+=(--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS")
-
-PLACEMENT_ARGS=(--device "$DEVICES" --split-mode "$SPLIT_MODE" --tensor-split "$TENSOR_SPLIT")
-
-MMPROJ_ARGS=()
-if [[ "$NO_MMPROJ" == "1" ]]; then
-    MMPROJ_ARGS=(--no-mmproj)
-else
-    # No autoload in this build: a server without the flag starts text-only
-    # and answers image requests with a 500, so name the projector.
-    MMPROJ_ARGS=(--mmproj "$MM_PROJ")
-fi
-exec "$LLAMA_BIN" \
-    --model "$MODEL" \
-    --alias "$ALIAS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --jinja \
-    --gpu-layers "$NGL" \
-    --fit "$FIT" \
-    --parallel "$PARALLEL" \
-    --ctx-size "$CTX_SIZE" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --flash-attn "$FLASH_ATTN" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --threads "$NUM_THREADS" \
-    --threads-batch "$THREAD_BATCH" \
-    --ctx-checkpoints "$CTX_CHECKPOINTS" \
-    --cache-ram "$CACHE_RAM" \
-    --cache-prompt \
-    --metrics \
-    --temp "$TEMPERATURE" \
-    --top-p "$TOP_P" \
-    --top-k "$TOP_K" \
-    --min-p "$MIN_P" \
-    --presence-penalty "$PRESENCE_PENALTY" \
-    --reasoning "$REASONING" \
-    --reasoning-format "$REASONING_FORMAT" \
-    --reasoning-budget "$REASONING_BUDGET" \
-    "${TEMPLATE_ARGS[@]}" \
-    "${PLACEMENT_ARGS[@]}" \
-    "${MMPROJ_ARGS[@]}" \
-    "${SPEC_ARGS[@]}"

scripts/.AGENTTRASH/bench-decode-threads-batch.py.strix-rocmfp4

@@ -1,192 +0,0 @@
-#!/usr/bin/env python3
-"""
-Decode throughput benchmark for frank-dragoon.
-Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
-generates 256 tokens, measures generation tok/s.
-
-Each --threads-batch value requires a cold server load (~3 min).
-"""
-import json, time, urllib.request, urllib.error, subprocess, os, sys
-
-MODEL = "/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"
-SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
-PORT = 7770
-N_GEN = 256
-PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
-ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
-SERVER_DIR = os.path.dirname(SERVE)
-ROCM_PATH = "/opt/rocm"
-
-def make_env():
-    """Build the same env as serve-frank-dragoon.sh."""
-    env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
-    
-    # Source env file for tunables
-    if os.path.exists(ENV_FILE):
-        with open(ENV_FILE) as f:
-            for line in f:
-                line = line.strip()
-                if line and not line.startswith("#") and "=" in line:
-                    k, v = line.split("=", 1)
-                    env[k] = v
-    
-    # Build LD_LIBRARY_PATH
-    lib_path = f"{SERVER_DIR}:{ROCM_PATH}/lib:{ROCM_PATH}/lib/rocm_sysdeps/lib:{ROCM_PATH}/lib/llvm/lib"
-    if os.environ.get("LD_LIBRARY_PATH"):
-        lib_path += f":{os.environ['LD_LIBRARY_PATH']}"
-    env["LD_LIBRARY_PATH"] = lib_path
-    env["ROCM_PATH"] = ROCM_PATH
-    
-    return env
-
-ENV = make_env()
-
-def kill_all():
-    subprocess.run(["pkill", "-9", "llama-server"], capture_output=True)
-    time.sleep(3)
-
-def wait_ready():
-    for i in range(600):
-        try:
-            r = urllib.request.urlopen(f"http://127.0.0.1:{PORT}/health", timeout=2)
-            if json.loads(r.read()).get("status") == "ok":
-                return True
-        except:
-            pass
-        time.sleep(1)
-    return False
-
-def send_request(prompt, max_tokens=N_GEN):
-    """Send chat completion, measure generation throughput."""
-    body = json.dumps({
-        "model": "frank-dragoon",
-        "messages": [{"role": "user", "content": prompt}],
-        "max_tokens": max_tokens,
-        "stream": False,
-        "temperature": 0.7,
-        "top_p": 0.8
-    }).encode()
-    
-    t0 = time.time()
-    req = urllib.request.Request(
-        f"http://127.0.0.1:{PORT}/v1/chat/completions",
-        data=body,
-        headers={"Content-Type": "application/json"},
-        method="POST"
-    )
-    r = urllib.request.urlopen(req, timeout=1200)
-    resp = json.loads(r.read())
-    wall = time.time() - t0
-    
-    usage = resp.get("usage", {})
-    pt = usage.get("prompt_tokens", 0)
-    gt = usage.get("completion_tokens", 0)
-    gtps = gt / wall if wall > 0 else 0
-    return pt, gt, wall, gtps
-
-def load_and_extend_prompt():
-    """Load prompt and extend to ~8K tokens if needed."""
-    with open(PROMPT_FILE) as f:
-        text = f.read()
-    
-    # Count approximate tokens (rough: ~4 chars per token for code)
-    approx_tokens = len(text) / 4.0
-    needed = 8200
-    
-    if approx_tokens < needed:
-        repeats = needed // max(int(approx_tokens), 1) + 1
-        text = (text + "\n") * repeats
-        words = text.split()[:needed]
-        text = " ".join(words)
-    
-    return text
-
-def main():
-    tb_values = [int(x) for x in sys.argv[1:]] if len(sys.argv) > 1 else [16, 32, 64, 128]
-    results = []
-    
-    prompt = load_and_extend_prompt()
-    n_words = len(prompt.split())
-    print(f"Prompt: {n_words} words (~{n_words} tokens), {len(prompt)} chars\n", flush=True)
-    
-    for tb in tb_values:
-        print(f"\n{'='*60}")
-        print(f"threads-batch = {tb}")
-        print(f"{'='*60}", flush=True)
-        
-        kill_all()
-        print("  Starting server...", flush=True)
-        
-        lib_path = ENV.get("LD_LIBRARY_PATH", "")
-        cmd = [
-            "/usr/bin/env",
-            "HSA_OVERRIDE_GFX_VERSION=11.5.1",
-            "HIP_VISIBLE_DEVICES=0",
-            f"LD_LIBRARY_PATH={lib_path}",
-            f"ROCM_PATH={ROCM_PATH}",
-            SERVE, "--model", MODEL, "--port", str(PORT), "--jinja",
-            "--device", "ROCm0", "--gpu-layers", "999", "--fit", "off",
-            "--threads", "16", "--threads-batch", str(tb),
-            "--parallel", "2", "--ctx-size", "131072",
-            "--batch-size", "4096", "--ubatch-size", "4096",
-            "--flash-attn", "on",
-            "--cache-type-k", "f16", "--cache-type-v", "f16",
-            "--cache-prompt", "--cache-idle-slots", "--metrics"
-        ]
-        
-        srv = subprocess.Popen(cmd, env=ENV, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
-        
-        ready = wait_ready()
-        if not ready:
-            print("  FAILED: server did not become ready", flush=True)
-            srv.terminate()
-            srv.wait()
-            for line in (srv.stdout.readlines() or [])[-20:]:
-                print(f"  LOG: {line.rstrip()}", flush=True)
-            continue
-        
-        print("  Server ready.", flush=True)
-        
-        # Warmup (small request)
-        try:
-            send_request("Hi", 16)
-        except Exception as e:
-            print(f"  Warmup failed: {e}", flush=True)
-            srv.terminate()
-            srv.wait()
-            continue
-        
-        # Benchmark
-        print(f"  Generating {N_GEN} tokens...", flush=True)
-        try:
-            pt, gt, wall, gtps = send_request(prompt, N_GEN)
-            print(f"  {gt}/{N_GEN} gen tokens in {wall:.1f}s = {gtps:.1f} tok/s")
-            print(f"  Prompt tokens: {pt}")
-            results.append({"tb": tb, "wall": wall, "gtps": gtps, "gt": gt, "pt": pt})
-        except Exception as e:
-            print(f"  FAILED: {e}", flush=True)
-        
-        srv.terminate()
-        srv.wait()
-        time.sleep(2)
-    
-    # Summary
-    print(f"\n{'='*60}")
-    print("SUMMARY")
-    print(f"{'='*60}")
-    for r in results:
-        print(f"  tb={r['tb']:>3}: {r['gt']} gen tokens, {r['wall']:.1f}s, {r['gtps']:.1f} tok/s")
-    
-    if results:
-        best = max(results, key=lambda r: r['gtps'])
-        print(f"\nBest: tb={best['tb']} at {best['gtps']:.1f} tok/s")
-    
-    with open("/tmp/bench_decode_results.txt", "w") as f:
-        f.write("tb,gen_tokens,wall_s,gen_tps,prompt_tokens\n")
-        for r in results:
-            f.write(f"{r['tb']},{r['gt']},{r['wall']:.1f},{r['gtps']:.1f},{r['pt']}\n")
-    
-    print(f"\nResults: /tmp/bench_decode_results.txt")
-
-if __name__ == "__main__":
-    main()

scripts/.AGENTTRASH/bench-ngram-mod.py.ik-llama.cpp

@@ -1,268 +0,0 @@
-#!/usr/bin/env python3
-"""
-Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
-
-Tests prompt-processing throughput with 16K token prompts (manageable request size).
-Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
-
-Usage:
-    # Generate prompts: python3 scripts/bench-ngram-mod.py generate /tmp/prompts/
-    # Run: python3 scripts/bench-ngram-mod.py /tmp/prompts/
-"""
-
-import json
-import os
-import sys
-import time
-import subprocess
-import urllib.request
-import urllib.error
-from pathlib import Path
-
-MODEL = "/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"
-CHAT_TEMPLATE = "/home/dank/serverAppConfigs/scripts/chat-template.jinja"
-BUILD_DIR = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
-LLAMA_BIN = os.path.join(BUILD_DIR, "llama-server")
-
-# 16K tokens ~= 64KB text
-TARGET_PROMPT_TOKENS = 16000
-TOKEN_CHAR_RATIO = 4
-TARGET_PROMPT_BYTES = TARGET_PROMPT_TOKENS * TOKEN_CHAR_RATIO
-
-CONTEXT = 65536
-BATCH = 4096
-UBATCH = 4096
-THREADS = 16
-THREAD_BATCH = 32
-NGRAM_N_MAX = 24
-NGRAM_N_MIN = 4
-TEST_TOKENS = 128
-SERVER_WAIT = 300
-
-
-def start_server(port, spec_type):
-    cmd = [
-        LLAMA_BIN,
-        "--model", MODEL,
-        "--chat-template-file", CHAT_TEMPLATE,
-        "--jinja",
-        "--device", "ROCm0",
-        "--gpu-layers", "999",
-        "--parallel", "1",
-        "--ctx-size", str(CONTEXT),
-        "--batch-size", str(BATCH),
-        "--ubatch-size", str(UBATCH),
-        "--fit", "off",
-        "--flash-attn", "on",
-        "--cache-type-k", "f16",
-        "--cache-type-v", "f16",
-        "--threads", str(THREADS),
-        "--threads-batch", str(THREAD_BATCH),
-        "--cache-prompt",
-        "--cache-idle-slots",
-        "--metrics",
-        "--host", "127.0.0.1",
-        "--port", str(port),
-        "--temp", "0",
-    ]
-    if spec_type == "ngram-mod":
-        cmd.extend([
-            "--spec-type", "ngram-mod",
-            "--spec-ngram-mod-n-max", str(NGRAM_N_MAX),
-            "--spec-ngram-mod-n-min", str(NGRAM_N_MIN),
-        ])
-    else:
-        cmd.extend(["--spec-type", "none"])
-
-    env = os.environ.copy()
-    env["ROCM_PATH"] = "/opt/rocm"
-    env["LD_LIBRARY_PATH"] = f"{BUILD_DIR}:/opt/rocm/lib:/opt/rocm/lib/rocm_sysdeps/lib:/opt/rocm/lib/llvm/lib"
-    env["HSA_OVERRIDE_GFX_VERSION"] = "11.5.1"
-    env["HIP_VISIBLE_DEVICES"] = "0"
-
-    proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-
-    print(f"  Waiting for server on port {port} (spec={spec_type})...")
-    for i in range(SERVER_WAIT):
-        try:
-            req = urllib.request.urlopen(f"http://127.0.0.1:{port}/health", timeout=2)
-            resp = json.loads(req.read().decode())
-            if resp.get("status") == "ok":
-                print(f"  Server ready after {i+1}s")
-                return proc
-        except:
-            time.sleep(1)
-
-    proc.terminate()
-    out, err = proc.communicate()
-    print(f"ERROR: Server failed to start in {SERVER_WAIT}s")
-    print(err.decode()[-2000:])
-    sys.exit(1)
-
-
-def run_test(prompt_text, spec_type, port):
-    """Send a request and extract prompt processing metrics."""
-    data = json.dumps({
-        "model": "frank-dragoon",
-        "messages": [{"role": "user", "content": prompt_text}],
-        "max_tokens": TEST_TOKENS,
-        "stream": False,
-        "temperature": 0,
-    }).encode()
-
-    req = urllib.request.Request(
-        f"http://127.0.0.1:{port}/v1/chat/completions",
-        data=data,
-        headers={"Content-Type": "application/json"},
-    )
-
-    start = time.monotonic()
-    resp = urllib.request.urlopen(req, timeout=600)
-    elapsed = time.monotonic() - start
-
-    result = json.loads(resp.read().decode())
-    timings = result.get("timings", {})
-    usage = result.get("usage", {})
-
-    return {
-        "prompt_tokens": usage.get("prompt_tokens", 0),
-        "prompt_ms": timings.get("prompt_ms", 0),
-        "prompt_per_second": timings.get("prompt_per_second", 0),
-        "prompt_per_token_ms": timings.get("prompt_per_token_ms", 0),
-        "predicted_per_second": timings.get("predicted_per_second", 0),
-        "total_time_s": elapsed,
-        "cache_n": timings.get("cache_n", 0),
-    }
-
-
-def main():
-    if len(sys.argv) < 2:
-        print(f"Usage: {sys.argv[0]} <prompt_dir>")
-        sys.exit(1)
-
-    prompt_dir = Path(sys.argv[1])
-    prompt_dir.mkdir(parents=True, exist_ok=True)
-
-    code_prompt_file = prompt_dir / "prompt_code.txt"
-    prose_prompt_file = prompt_dir / "prompt_prose.txt"
-
-    # Generate prompts if they don't exist
-    if not code_prompt_file.exists() or not prose_prompt_file.exists():
-        print("Generating prompts...")
-        # Generate deterministic prompts
-        # Code prompt: structured algorithms + data generation + config schemas
-        code_lines = []
-        for i in range(50):
-            code_lines.append(f"# Algorithm {i}\n")
-            code_lines.append(f"def algo_{i}(x, y):\n")
-            code_lines.append(f"    \"\"\"Algorithm {i} processing.\"\"\"\n")
-            code_lines.append(f"    result = 0\n")
-            code_lines.append(f"    for j in range({i * 10}):\n")
-            code_lines.append(f"        result += x * y + j\n")
-            code_lines.append(f"    return result\n\n")
-        code_text = "".join(code_lines)
-        # Pad/truncate to target
-        while len(code_text) < TARGET_PROMPT_BYTES:
-            code_text += "# Additional algorithm block\n"
-            code_text += f"def extra_{len(code_text)//100}():\n"
-            code_text += "    x = 1\n"
-            code_text += "    y = 2\n"
-            code_text += "    return x + y\n\n"
-        code_text = code_text[:TARGET_PROMPT_BYTES]
-
-        # Prose prompt: historical/scientific/philosophical text
-        prose_text = ""
-        topics = [
-            "The Renaissance brought profound transformation to European civilization. Artists and scholars pushed the boundaries of human expression.",
-            "Quantum mechanics describes the behavior of matter and energy at atomic scales. The uncertainty principle limits measurement precision.",
-            "Ethical philosophy grapples with questions of right and wrong. Utilitarianism judges actions by their consequences and the happiness they produce.",
-            "Mountains rise like ancient sentinels, their peaks lost in clouds that swirl and dance across the sky above the valleys below.",
-            "The ocean whispers secrets to the shore, each wave carrying fragments of distant lands across the water before retreating back to sea.",
-        ]
-        for _ in range(500):
-            prose_text += f"{topics[len(prose_text) % len(topics)]} "
-        prose_text = prose_text[:TARGET_PROMPT_BYTES]
-
-        code_prompt_file.write_text(code_text)
-        prose_prompt_file.write_text(prose_text)
-        print(f"  Code: {code_prompt_file.stat().st_size:,} bytes")
-        print(f"  Prose: {prose_prompt_file.stat().st_size:,} bytes")
-
-    print("=" * 60)
-    print("  N-gram Mod Benchmark — frank-dragoon")
-    print(f"  STRIX ROCmFP4 · 64K context · 16K prompts")
-    print("=" * 60)
-
-    results = []
-    port_base = 7772
-    port_ngram = 7773
-
-    tests = [
-        ("code", "none", port_base, code_prompt_file),
-        ("prose", "none", port_ngram, prose_prompt_file),
-        ("code", "ngram-mod", port_base, code_prompt_file),
-        ("prose", "ngram-mod", port_ngram, prose_prompt_file),
-    ]
-
-    for prompt_type, spec_type, port, pfile in tests:
-        print(f"\n=== {spec_type} ({prompt_type}) ===")
-        print(f"  File: {pfile} ({pfile.stat().st_size:,} bytes)")
-        print(f"  Spec: {spec_type}")
-        if spec_type == "ngram-mod":
-            print(f"  n_max={NGRAM_N_MAX}, n_min={NGRAM_N_MIN}")
-
-        proc = start_server(port, spec_type)
-
-        try:
-            # Warm-up
-            print("  Warm-up...")
-            run_test("Hello test", spec_type, port)
-
-            # Full prompt
-            print(f"  Sending {TARGET_PROMPT_TOKENS} token prompt...")
-            with open(pfile) as f:
-                prompt_text = f.read()
-            result = run_test(prompt_text, spec_type, port)
-
-            tps = result["prompt_per_second"]
-            print(f"  Tokens:  {result['prompt_tokens']}")
-            print(f"  Prompt:  {result['prompt_ms']:.0f}ms")
-            print(f"  TPS:     {tps:.1f} tok/s")
-            print(f"  Total:   {result['total_time_s']:.1f}s")
-
-            results.append({
-                "prompt_type": prompt_type,
-                "spec_type": spec_type,
-                "prompt_tokens": result["prompt_tokens"],
-                "prompt_ms": result["prompt_ms"],
-                "tps": tps,
-                "total_time_s": result["total_time_s"],
-            })
-
-        finally:
-            proc.terminate()
-            try:
-                proc.wait(timeout=5)
-            except:
-                proc.kill()
-
-    # Summary
-    print("\n" + "=" * 60)
-    print("  RESULTS SUMMARY")
-    print("=" * 60)
-
-    for ptype in ["code", "prose"]:
-        baseline = next((r for r in results if r["prompt_type"] == ptype and r["spec_type"] == "none"), None)
-        ngram = next((r for r in results if r["prompt_type"] == ptype and r["spec_type"] == "ngram-mod"), None)
-        if baseline and ngram:
-            gain = ((ngram["tps"] - baseline["tps"]) / baseline["tps"]) * 100 if baseline["tps"] > 0 else 0
-            print(f"\n{ptype.upper()} PROMPT:")
-            print(f"  Baseline TPS: {baseline['tps']:.1f} tok/s ({baseline['prompt_ms']:.0f}ms)")
-            print(f"  N-gram TPS:   {ngram['tps']:.1f} tok/s ({ngram['prompt_ms']:.0f}ms)")
-            print(f"  Speedup:      {gain:+.1f}%")
-
-    print("\n" + "=" * 60)
-
-
-if __name__ == "__main__":
-    main()

scripts/.AGENTTRASH/bench-ngram-mod.sh.ik-llama.cpp

@@ -1,304 +0,0 @@
-#!/usr/bin/env bash
-# Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
-#
-# This tests prompt-processing throughput at 64k context with two distinct
-# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
-# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
-#
-# Run on Dankclaw after stopping the production service:
-#   sudo systemctl stop frank-dragoon
-#   ./scripts/bench-ngram-mod.sh
-#   sudo systemctl start frank-dragoon
-
-set -euo pipefail
-
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a; source "$ENV_FILE"; set +a
-fi
-
-: "${LLAMA_BIN:=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
-: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
-: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
-: "${CHAT_TEMPLATE_KWARGS:={}}"
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=4096}"
-: "${CTX_SIZE:=65536}"
-: "${NUM_THREADS:=16}"
-: "${THREAD_BATCH:=32}"
-: "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-: "${FIT:=off}"
-: "${TEMPERATURE:=0.7}"
-: "${TOP_P:=0.8}"
-: "${TOP_K:=20}"
-: "${MIN_P:=0.0}"
-: "${REASONING:=on}"
-: "${REASONING_FORMAT:=deepseek}"
-
-MODEL="$MODEL_DIR/$MODEL_FILE"
-HOSTNAME=$(hostname 2>/dev/null || echo "dankclaw")
-
-# Prompt files (generated by generate-prompts.py or passed via args)
-PROMPT_CODE="${1:-}"
-PROMPT_PROSE="${2:-}"
-
-if [[ -z "$PROMPT_CODE" || -z "$PROMPT_PROSE" ]]; then
-    echo "ERROR: Need two prompt file paths."
-    echo "Usage: $0 <prompt_code.txt> <prompt_prose.txt>"
-    echo "Generate with: python3 scripts/generate-prompts.py"
-    exit 1
-fi
-
-if [[ ! -r "$MODEL" ]]; then
-    echo "missing or unreadable model: $MODEL" >&2
-    exit 1
-fi
-
-SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
-ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
-LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
-
-# Scratch ports
-PORT_NGRAM=7771
-PORT_BASELINE=7772
-
-cleanup() {
-    echo "Cleaning up servers..."
-    # Kill any leftover llama-server on our ports
-    for port in $PORT_NGRAM $PORT_BASELINE; do
-        fuser -k "$port"/tcp 2>/dev/null || true
-    done
-}
-trap cleanup EXIT
-
-wait_for_server() {
-    local port=$1
-    local max_wait=120
-    local elapsed=0
-    while (( elapsed < max_wait )); do
-        if curl -s "http://127.0.0.1:$port/health" | grep -q '"running"'; then
-            return 0
-        fi
-        sleep 1
-        (( elapsed++ ))
-    done
-    echo "ERROR: Server on port $port did not start within ${max_wait}s" >&2
-    return 1
-}
-
-get_prompt_tokens() {
-    local file=$1
-    # Count tokens approximately (1 token ~ 4 chars for English/code)
-    wc -c < "$file" | awk '{printf "%d", $1/4}'
-}
-
-run_single_test() {
-    local port=$1
-    local spec_type="$2"
-    local prompt_file=$3
-    local label=$4
-    local ngram_n_max="${5:-24}"
-    local ngram_n_min="${6:-4}"
-    
-    echo "=== $label ==="
-    echo "  spec-type: $spec_type"
-    echo "  ngram-mod n-max: $ngram_n_max (if enabled)"
-    echo "  ngram-mod n-min: $ngram_n_min (if enabled)"
-    echo "  prompt: $prompt_file ($(wc -c < "$prompt_file") bytes)"
-    
-    # Build command
-    local cmd=(
-        /usr/bin/env
-        ROCM_PATH="$ROCM_PATH"
-        HIP_VISIBLE_DEVICES=0
-        LD_LIBRARY_PATH="$LIBRARY_PATH"
-        HSA_OVERRIDE_GFX_VERSION="${HSA_OVERRIDE_GFX_VERSION:-11.5.1}"
-        "$LLAMA_BIN"
-        --model "$MODEL"
-        --chat-template-file "$CHAT_TEMPLATE_FILE"
-        --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS"
-        --host 127.0.0.1
-        --port "$port"
-        --jinja
-        --device ROCm0
-        --gpu-layers 999
-        --parallel 1
-        --ctx-size "$CTX_SIZE"
-        --batch-size "$BATCH_SIZE"
-        --ubatch-size "$UBATCH_SIZE"
-        --fit "$FIT"
-        --flash-attn on
-        --cache-type-k "$CACHE_TYPE_K"
-        --cache-type-v "$CACHE_TYPE_V"
-        --threads "$NUM_THREADS"
-        --threads-batch "$THREAD_BATCH"
-        --cache-prompt
-        --cache-idle-slots
-        --metrics
-        --temp "$TEMPERATURE"
-        --top-p "$TOP_P"
-        --top-k "$TOP_K"
-        --min-p "$MIN_P"
-        --reasoning "$REASONING"
-        --reasoning-format "$REASONING_FORMAT"
-        --reasoning-budget -1
-    )
-    
-    # Add spec flags
-    if [[ "$spec_type" == "ngram-mod" ]]; then
-        cmd+=(
-            --spec-type "ngram-mod"
-            --spec-ngram-mod-n-max "$ngram_n_max"
-            --spec-ngram-mod-n-min "$ngram_n_min"
-        )
-    else
-        cmd+=(
-            --spec-type "none"
-        )
-    fi
-    
-    # Start server with stderr to log
-    local log_file="/tmp/ngram-bench-${label//[^a-zA-Z0-9]/-}.log"
-    echo "  Starting server (log: $log_file)..."
-    
-    "${cmd[@]}" 2> "$log_file" &
-    local server_pid=$!
-    
-    if ! wait_for_server "$port"; then
-        echo "ERROR: Server failed to start" >&2
-        kill "$server_pid" 2>/dev/null || true
-        return 1
-    fi
-    
-    echo "  Server ready."
-    
-    # Send prompt via API
-    echo "  Sending prompt..."
-    local prompt_text
-    prompt_text=$(cat "$prompt_file")
-    
-    local response
-    response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
-        -H "Content-Type: application/json" \
-        -d "{
-            \"model\": \"frank-dragoon\",
-            \"prompt\": \"$prompt_text\",
-            \"max_tokens\": 128,
-            \"temperature\": 0,
-            \"stream\": false
-        }")
-    
-    # Extract metrics
-    local prompt_tokens prompt_eval_duration_ms total_duration_ms
-    prompt_tokens=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['prompt_eval_count'])" 2>/dev/null || echo "0")
-    prompt_eval_duration_ms=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['prompt_eval_duration_ms'])" 2>/dev/null || echo "0")
-    total_duration_ms=$(echo "$response" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['total_duration_ms'])" 2>/dev/null || echo "0")
-    
-    # Calculate throughput
-    local prompt_tps=0
-    if [[ "$prompt_eval_duration_ms" -gt 0 ]]; then
-        prompt_tps=$(python3 -c "print(f'{($prompt_tokens / $prompt_eval_duration_ms) * 1000:.1f}')")
-    fi
-    
-    echo "  Prompt tokens:  $prompt_tokens"
-    echo "  Prompt eval:    ${prompt_eval_duration_ms}ms"
-    echo "  Prompt TPS:     $prompt_tps tok/s"
-    echo "  Total duration: ${total_duration_ms}ms"
-    
-    # Kill server
-    kill "$server_pid" 2>/dev/null || true
-    sleep 2
-    
-    echo "  Done."
-    echo "---"
-    
-    # Output structured results for parsing
-    echo "RESULT|$label|$prompt_file|$prompt_tokens|$prompt_eval_duration_ms|$total_duration_ms|$prompt_tps"
-}
-
-echo "╔══════════════════════════════════════════════════════════╗"
-echo "║  N-gram Mod Benchmark — frank-dragoon             ║"
-echo "║  STRIX ROCmFP4 · 64k context · two prompts             ║"
-echo "╚══════════════════════════════════════════════════════════╝"
-echo ""
-echo "Host: $HOSTNAME"
-echo "Model: $MODEL"
-echo "Build: $(basename "$LLAMA_BIN")"
-echo "Context: $CTX_SIZE"
-echo ""
-
-# Run baseline (no spec)
-echo "=== BASELINE (no speculative decoding) ==="
-echo ""
-
-BASELINE_CODE_RESULT=$(run_single_test $PORT_BASELINE "none" "$PROMPT_CODE" "Baseline-Code" 24 4)
-BASELINE_PROSE_RESULT=$(run_single_test $PORT_NGRAM "none" "$PROMPT_PROSE" "Baseline-Prose" 24 4)
-
-echo ""
-echo "=== NGRAM-MOD (speculative decoding enabled) ==="
-echo ""
-
-# Run n-gram mod
-NGRAM_CODE_RESULT=$(run_single_test $PORT_NGRAM "ngram-mod" "$PROMPT_CODE" "Ngram-Code" 24 4)
-NGRAM_PROSE_RESULT=$(run_single_test $PORT_BASELINE "ngram-mod" "$PROMPT_PROSE" "Ngram-Prose" 24 4)
-
-echo ""
-echo "╔══════════════════════════════════════════════════════════╗"
-echo "║  RESULTS SUMMARY                                       ║"
-echo "╚══════════════════════════════════════════════════════════╝"
-echo ""
-
-# Parse and display results
-python3 << 'PYEOF'
-import sys
-
-results = []
-for line in sys.stdin:
-    line = line.strip()
-    if not line.startswith("RESULT|"):
-        continue
-    parts = line.split("|")
-    if len(parts) != 8:
-        continue
-    label, prompt_file, prompt_tokens, prompt_eval_ms, total_ms, tps = parts[1], parts[2], int(parts[3]), int(parts[4]), int(parts[5]), float(parts[6])
-    results.append({
-        'label': label,
-        'prompt_file': prompt_file,
-        'prompt_tokens': prompt_tokens,
-        'prompt_eval_ms': prompt_eval_ms,
-        'total_ms': total_ms,
-        'tps': tps
-    })
-
-# Compare baseline vs ngram for each prompt type
-for code_result in results:
-    if not code_result['label'].startswith('Ngram-'):
-        continue
-    prompt_type = code_result['label'].replace('Ngram-', '').replace('Ngram', 'Baseline').replace('-', '-')
-    baseline = next((r for r in results if r['label'] == 'Baseline-' + prompt_type), None)
-    if not baseline:
-        continue
-    
-    spec = code_result['label'] == 'Ngram-' + prompt_type
-    
-    print(f"Prompt: {prompt_type}")
-    print(f"  Tokens:   {code_result['prompt_tokens']}")
-    if spec:
-        print(f"  Spec:     ngram-mod (n_max=24, n_min=4)")
-        print(f"  Prompt:   {code_result['prompt_eval_ms']}ms  |  {code_result['tps']:>10.1f} tok/s")
-        baseline_tps = baseline['tps']
-        if baseline_tps > 0:
-            improvement = ((code_result['tps'] - baseline_tps) / baseline_tps) * 100
-            print(f"  Speedup:  {improvement:+.1f}% vs baseline ({baseline_tps:.1f} tok/s)")
-    else:
-        print(f"  Spec:     none (baseline)")
-        print(f"  Prompt:   {baseline['prompt_eval_ms']}ms  |  {baseline['tps']:>10.1f} tok/s")
-    print()
-PYEOF
-
-echo ""
-echo "Raw data lines above (RESULT|... format) can be parsed for automation."
-echo ""

scripts/.AGENTTRASH/bench-spec-decode.py.ik-llama.cpp

@@ -1,229 +0,0 @@
-#!/usr/bin/env python3
-"""Benchmark decode throughput across speculative decoding configs.
-
-Sweeps 4 configs: none, ngram-mod, draft-mtp, draft-mtp+ngram-mod.
-Each config generates 16k tokens from a minimal prompt at temp 0.
-
-Runs ON the target host (Dankclaw). Service management uses local sudo.
-Usage:
-    python3 bench-spec-decode.py [--max-tokens N] [--host H] [--port P]
-"""
-import json
-import subprocess
-import sys
-import time
-import urllib.request
-import urllib.error
-
-SERVICE = "frank-dragoon"
-ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
-ALIAS = "frank-dragoon"
-DEFAULT_HOST = "localhost"
-DEFAULT_PORT = 7770
-DEFAULT_MAX_TOKENS = 16384
-HEALTH_TIMEOUT = 1200  # 20 min for cold load
-HEALTH_INTERVAL = 5
-
-PROMPT = (
-    "Write a comprehensive, detailed essay about the complete history of "
-    "computing, from the earliest mechanical calculators through modern AI. "
-    "Cover every major era in detail, including key figures, machines, and "
-    "breakthroughs. Be as thorough as possible."
-)
-
-CONFIGS = [
-    ("none", "none"),
-    ("ngram-mod", "ngram-mod"),
-    ("draft-mtp", "draft-mtp"),
-    ("draft-mtp+ngram-mod", "draft-mtp,ngram-mod"),
-]
-
-
-def local(cmd):
-    """Run a command locally (the script runs on the target host)."""
-    r = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=120)
-    if r.returncode != 0:
-        print(f"  WARNING: {cmd} -> exit {r.returncode}: {r.stderr.strip()}",
-              file=sys.stderr)
-    return r
-
-
-def sed_spec_type(value):
-    local(f"sudo sed -i 's/^SPEC_TYPE=.*/SPEC_TYPE={value}/' {ENV_FILE}")
-
-
-def wait_health(host, port, timeout=HEALTH_TIMEOUT):
-    start = time.time()
-    while time.time() - start < timeout:
-        try:
-            resp = urllib.request.urlopen(
-                f"http://{host}:{port}/health", timeout=10
-            )
-            if resp.status == 200:
-                return True
-        except (urllib.error.URLError, urllib.error.HTTPError, ConnectionError,
-                OSError):
-            pass
-        time.sleep(HEALTH_INTERVAL)
-    return False
-
-
-def benchmark(host, port, alias, max_tokens):
-    body = json.dumps({
-        "model": alias,
-        "messages": [{"role": "user", "content": PROMPT}],
-        "max_tokens": max_tokens,
-        "temperature": 0,
-        "stream": False,
-    }).encode()
-    req = urllib.request.Request(
-        f"http://{host}:{port}/v1/chat/completions",
-        data=body,
-        headers={"Content-Type": "application/json"},
-    )
-    t0 = time.time()
-    resp = urllib.request.urlopen(req, timeout=1200)
-    wall = time.time() - t0
-    data = json.loads(resp.read())
-    return data, wall
-
-
-def main():
-    import argparse
-    ap = argparse.ArgumentParser(description=__doc__)
-    ap.add_argument("--host", default=DEFAULT_HOST)
-    ap.add_argument("--port", type=int, default=DEFAULT_PORT)
-    ap.add_argument("--max-tokens", type=int, default=DEFAULT_MAX_TOKENS)
-    args = ap.parse_args()
-
-    # Read original SPEC_TYPE from the env file
-    r = local(f"grep '^SPEC_TYPE=' {ENV_FILE}")
-    original_spec = r.stdout.strip()
-    orig_val = (original_spec.split("=", 1)[1]
-                if "=" in original_spec else "draft-mtp,ngram-mod")
-    print(f"Original: {original_spec}")
-    print(f"Max tokens: {args.max_tokens}")
-    print(f"Configs: {[c[0] for c in CONFIGS]}")
-    print()
-
-    # Stop + mask the production service
-    print("Stopping + masking production service...")
-    local(f"sudo systemctl stop {SERVICE}")
-    local(f"sudo systemctl mask {SERVICE}")
-    time.sleep(2)
-
-    results = []
-    for name, spec_type in CONFIGS:
-        print(f"\n{'='*60}")
-        print(f"Config: {name} (SPEC_TYPE={spec_type})")
-        print(f"{'='*60}")
-
-        # Set SPEC_TYPE
-        sed_spec_type(spec_type)
-        print(f"Set SPEC_TYPE={spec_type}")
-
-        # Start the service
-        local(f"sudo systemctl unmask {SERVICE}")
-        local(f"sudo systemctl start {SERVICE}")
-        print("Starting service...")
-
-        # Wait for health
-        print(f"Waiting for health (up to {HEALTH_TIMEOUT}s)...")
-        if not wait_health(args.host, args.port):
-            print(f"FAILED: service did not become healthy for {name}")
-            results.append({
-                "config": name, "spec_type": spec_type,
-                "error": "health timeout",
-            })
-            local(f"sudo systemctl stop {SERVICE}")
-            local(f"sudo systemctl mask {SERVICE}")
-            continue
-
-        print("Healthy. Running benchmark...")
-
-        # Run benchmark
-        try:
-            data, wall = benchmark(
-                args.host, args.port, ALIAS, args.max_tokens
-            )
-        except Exception as e:
-            print(f"FAILED: {e}")
-            results.append({
-                "config": name, "spec_type": spec_type,
-                "error": str(e),
-            })
-            local(f"sudo systemctl stop {SERVICE}")
-            local(f"sudo systemctl mask {SERVICE}")
-            continue
-
-        timings = data.get("timings", {})
-        usage = data.get("usage", {})
-        pred_n = timings.get("predicted_n", usage.get("completion_tokens", 0))
-        pred_ms = timings.get("predicted_ms", 0)
-        pred_ps = timings.get("predicted_per_second", 0)
-        prompt_n = timings.get("prompt_n", usage.get("prompt_tokens", 0))
-        prompt_ms = timings.get("prompt_ms", 0)
-
-        print(f"  Generated:   {pred_n} tokens")
-        print(f"  Wall time:   {wall:.1f}s")
-        print(f"  Decode:      {pred_ps:.2f} tok/s")
-        print(f"  Prefill:     {prompt_n} tokens in {prompt_ms:.0f}ms"
-              f" ({prompt_n / max(prompt_ms, 1) * 1000:.1f} tok/s)")
-
-        results.append({
-            "config": name,
-            "spec_type": spec_type,
-            "predicted_tokens": pred_n,
-            "predicted_ms": pred_ms,
-            "predicted_per_second": pred_ps,
-            "prompt_tokens": prompt_n,
-            "prompt_ms": prompt_ms,
-            "wall_time": wall,
-        })
-
-        # Stop + mask between configs
-        local(f"sudo systemctl stop {SERVICE}")
-        local(f"sudo systemctl mask {SERVICE}")
-        time.sleep(2)
-
-    # Restore original SPEC_TYPE
-    sed_spec_type(orig_val)
-    print(f"\nRestoring SPEC_TYPE={orig_val}")
-    print("Unmasking + starting production service...")
-    local(f"sudo systemctl unmask {SERVICE}")
-    local(f"sudo systemctl start {SERVICE}")
-
-    # Health check
-    if wait_health(args.host, args.port):
-        print("Production service healthy.")
-    else:
-        print("WARNING: production service not healthy after restore!")
-
-    # Print results table
-    print(f"\n{'='*70}")
-    print("RESULTS")
-    print(f"{'='*70}")
-    print(f"{'Config':<30} {'Tokens':>8} {'Wall(s)':>10} {'Decode tok/s':>14} "
-          f"{'Error':>20}")
-    print(f"{'-'*30} {'-'*8} {'-'*10} {'-'*14} {'-'*20}")
-    for r in results:
-        if "error" in r:
-            print(f"{r['config']:<30} {'':>8} {'':>10} {'':>14} "
-                  f"{r['error']:>20}")
-        else:
-            print(f"{r['config']:<30} {r['predicted_tokens']:>8} "
-                  f"{r['wall_time']:>10.1f} {r['predicted_per_second']:>14.2f} "
-                  f"{'':>20}")
-    print(f"{'='*70}")
-
-    # Save JSON results
-    ts = time.strftime("%Y%m%d-%H%M%S")
-    out = f"/tmp/bench-spec-decode-{ts}.json"
-    with open(out, "w") as f:
-        json.dump({"results": results, "max_tokens": args.max_tokens,
-                   "timestamp": ts}, f, indent=2)
-    print(f"Results saved: {out}")
-
-
-if __name__ == "__main__":
-    main()

scripts/.AGENTTRASH/bench-vllm-corpus.py

@@ -1,85 +0,0 @@
-#!/usr/bin/env python3
-"""bench-vllm-corpus.py — build a code-shaped benchmark corpus from the repo.
-
-Why this exists: `vllm bench serve --dataset-name random` produces synthetic
-token soup, and speculative decoding accepts it nothing like real code — measured
-on frank-medium, random prompts accepted 2.9-3.6 tokens/update against the ~5.8
-the upstream reference reports on a code mix, which makes the random corpus
-under-report throughput by a wide margin and would pick the wrong SPEC depth.
-
-The prompts are the shape agents actually send: a repository artifact plus an
-instruction that requires reading it. Output length is fixed at bench time
-(`--custom-output-len`), so the corpus carries prompts only.
-
-Usage:
-    bench-vllm-corpus.py [repo-root] [out.jsonl]
-
-Then feed it to the endpoint (see scripts/bench-vllm-serve.sh, which passes the
-tokenizer the client needs and salts prompts so no run inherits a warm prefix
-cache). Methodology and the numbers these corpora produced are in
-serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md.
-"""
-
-from __future__ import annotations
-
-import json
-import pathlib
-import sys
-
-REPO = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else "/home/dank/serverAppConfigs")
-OUT = pathlib.Path(sys.argv[2] if len(sys.argv) > 2 else "/home/dank/bench/corpus-code.jsonl")
-
-TARGETS = [
-    "scripts/deploy-remote.sh",
-    "scripts/deploy-plan.py",
-    "scripts/deploy.sh",
-    "dankserver/frank-medium/radiance-entrypoint.sh",
-    "dankserver/frank-medium/quadlet/frank-medium-a.container",
-    "dankserver/frank-medium/proxy/affinity-proxy.py",
-    "dankserver/frank-medium/proxy/routing.py",
-    "tests/frank_medium_quadlet.bats",
-    "tests/deployment_scoping.bats",
-    "dankclaw/hermes/quadlet/hermes.container",
-    "dankclaw/anubis/quadlet/anubis.container",
-    "shared/prometheus/prometheus.yml",
-    "shared/prometheus/rules/llm-fleet.yml",
-    "shared/loki/loki-config.yaml",
-    "dankclaw/compose.yaml",
-    "dankserver/copyparty/quadlet/copyparty.container",
-]
-
-PROMPTS = [
-    "Read this repository artifact and summarise what it does, then list every "
-    "assumption it makes about the host it runs on.\n\n```\n{body}\n```",
-    "Here is a file from our homelab repository. Identify any bug, race or "
-    "missing error path, and show the minimal patch that fixes it.\n\n```\n{body}\n```",
-    "Explain this configuration to a new operator, then rewrite it with "
-    "comments that state the constraint each line protects.\n\n```\n{body}\n```",
-    "This file changed in a recent deploy. Write the changelog entry and a test "
-    "that would fail if the change regressed.\n\n```\n{body}\n```",
-]
-
-MAX_LINES = 220
-
-
-def main() -> int:
-    samples = []
-    for i, rel in enumerate(TARGETS):
-        path = REPO / rel
-        if not path.is_file():
-            continue
-        body = "\n".join(path.read_text(errors="replace").splitlines()[:MAX_LINES])
-        samples.append({"prompt": PROMPTS[i % len(PROMPTS)].format(body=body)})
-    if not samples:
-        print(f"no corpus sources found under {REPO}", file=sys.stderr)
-        return 1
-    OUT.parent.mkdir(parents=True, exist_ok=True)
-    with OUT.open("w") as fh:
-        for sample in samples:
-            fh.write(json.dumps(sample) + "\n")
-    print(f"wrote {len(samples)} prompts to {OUT}")
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())

scripts/.AGENTTRASH/bench-vllm-serve.sh

@@ -1,63 +0,0 @@
-#!/bin/bash
-# bench-vllm-serve.sh — concurrency sweep against a vLLM endpoint.
-#
-# Usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]
-#   e.g. bench-vllm-serve.sh baseline 7770 1 4 8
-#
-# Runs `vllm bench serve` from inside the serving image (so the client matches
-# the server build) through an OpenAI-compatible endpoint, on the code corpus
-# scripts/bench-vllm-corpus.py writes.
-#
-# Two traps this script exists to encode, both of which produced silent garbage
-# during the 2026-09-16 frank-medium work:
-#
-#   1. The tokenizer must be passed explicitly and /models mounted. With only
-#      `--model frank-medium` the client tries to resolve that alias on
-#      HuggingFace and dies before sending a request — a run that looks like a
-#      benchmark and reports nothing.
-#   2. `--ignore-eos` is deliberate: it measures decode throughput at a fixed
-#      output length. Drop it when you care about real stop behaviour, but then
-#      do not compare the numbers with anything measured under it.
-#
-# Also note the corpora are code-shaped on purpose (see the generator's header):
-# synthetic random prompts misreport speculative acceptance badly.
-#
-# Methodology, and the numbers these runs produced on 2x R9700, are in
-# serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md.
-set -u
-
-TAG=${1:?usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]}
-PORT=${2:?usage: bench-vllm-serve.sh <tag> <port> [concurrency ...]}
-shift 2
-CONCS=("$@")
-[ "${#CONCS[@]}" -eq 0 ] && CONCS=(1 4 8)
-
-IMG=${IMG:-localhost/radiance-vllm:0.9.3}
-MODELS_DIR=${MODELS_DIR:-/home/dank/models}
-TOK=${TOK:-/models/Qwen3.8-27B-Uncensored-MXFP4-awq}
-BENCH_DIR=${BENCH_DIR:-/home/dank/bench}
-CORPUS=${CORPUS:-$BENCH_DIR/corpus-code.jsonl}
-OUT=${OUT:-$BENCH_DIR/$TAG}
-mkdir -p "$OUT"
-
-if [ ! -f "$CORPUS" ]; then
-    echo "corpus $CORPUS is missing; run scripts/bench-vllm-corpus.py first" >&2
-    exit 1
-fi
-
-for c in "${CONCS[@]}"; do
-    echo "== $TAG conc=$c (port $PORT)"
-    podman run --rm --network=host -v "$MODELS_DIR":/models:ro -v "$BENCH_DIR":/bench:ro \
-        -e HF_HUB_OFFLINE=1 --entrypoint bash "$IMG" -lc "
-        vllm bench serve --backend openai-chat \
-          --base-url http://127.0.0.1:$PORT --endpoint /v1/chat/completions \
-          --model frank-medium --tokenizer $TOK \
-          --dataset-name custom --dataset-path /bench/$(basename "$CORPUS") \
-          --custom-output-len 512 --num-prompts 16 --max-concurrency $c \
-          --disable-shuffle --seed 0 --ignore-eos \
-          --percentile-metrics ttft,tpot --metric-percentiles 50,99 \
-          --disable-tqdm 2>&1 | grep -E 'Successful requests|Benchmark duration|Output token throughput|Total Token throughput|Mean TTFT|P50 TTFT|P50 TPOT|Acceptance rate|Acceptance length'
-    " 2>&1 | sed 's/^/  /'
-    echo
-done
-echo "results for $TAG written under $OUT"

scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp

@@ -1,453 +0,0 @@
-{%- set template_version = "qwen3.8-froggeric-v22.4.0" %}
-{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
-{%- set image_count = namespace(value=0) %}
-{%- set video_count = namespace(value=0) %}
-{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
-{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
-{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
-{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
-    {%- set _preserve_thinking = preserve_reasoning %}
-{%- elif preserve_thinking is defined and preserve_thinking is not none %}
-    {%- set _preserve_thinking = preserve_thinking %}
-{%- else %}
-    {%- set _preserve_thinking = true %}
-{%- endif %}
-{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
-{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
-{%- set _default_reasoning_effort = 'medium' %}
-{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
-{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}
-{%- set _initial_thinking = enable_thinking %}
-{%- if _effort_raw in ('none', 'off') %}
-    {%- set _initial_thinking = false %}
-    {%- set _initial_effort = 'medium' %}
-{%- elif _effort_raw in ('minimal', 'low') %}
-    {%- set _initial_effort = 'low' %}
-{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
-    {%- set _initial_effort = 'xhigh' %}
-{%- else %}
-    {%- set _initial_effort = 'medium' %}
-{%- endif %}
-{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
-{%- if auto_disable_thinking_with_tools and _has_tools %}
-    {%- set ns_state.thinking = false %}
-{%- endif %}
-{%- for msg in messages %}
-    {%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
-        {%- if msg.content is string %}
-            {%- if '<|think_off|>' in msg.content %}
-                {%- set ns_state.thinking = false %}
-            {%- elif '<|think_on|>' in msg.content %}
-                {%- set ns_state.thinking = true %}
-            {%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
-                {%- set ns_state.thinking = true %}
-                {%- set ns_state.effort = 'xhigh' %}
-            {%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
-                {%- set ns_state.thinking = true %}
-                {%- set ns_state.effort = 'low' %}
-            {%- elif '<|think_medium|>' in msg.content %}
-                {%- set ns_state.thinking = true %}
-                {%- set ns_state.effort = 'medium' %}
-            {%- endif %}
-        {%- elif msg.content is iterable and msg.content is not mapping %}
-            {%- for item in msg.content %}
-                {%- if item is string %}
-                    {%- set _item_text = item %}
-                {%- elif item is mapping and 'text' in item and item.text is string %}
-                    {%- set _item_text = item.text %}
-                {%- else %}
-                    {%- set _item_text = '' %}
-                {%- endif %}
-                {%- if _item_text %}
-                    {%- if '<|think_off|>' in _item_text %}
-                        {%- set ns_state.thinking = false %}
-                    {%- elif '<|think_on|>' in _item_text %}
-                        {%- set ns_state.thinking = true %}
-                    {%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
-                        {%- set ns_state.thinking = true %}
-                        {%- set ns_state.effort = 'xhigh' %}
-                    {%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
-                        {%- set ns_state.thinking = true %}
-                        {%- set ns_state.effort = 'low' %}
-                    {%- elif '<|think_medium|>' in _item_text %}
-                        {%- set ns_state.thinking = true %}
-                        {%- set ns_state.effort = 'medium' %}
-                    {%- endif %}
-                {%- endif %}
-            {%- endfor %}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- set reasoning_instructions = '' %}
-{%- if ns_state.thinking %}
-    {%- if ns_state.effort == 'xhigh' %}
-        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
-    {%- elif ns_state.effort == 'low' %}
-        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
-    {%- endif %}
-{%- endif %}
-{%- macro render_content(content, do_vision_count, is_system_content=false) %}
-    {%- if content is string %}
-        {{- content }}
-    {%- elif content is iterable and content is not mapping %}
-        {%- for item in content %}
-            {%- if item is mapping %}
-                {%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
-                    {%- if is_system_content %}
-                        {{- raise_exception('System message cannot contain images.') }}
-                    {%- endif %}
-                    {%- if do_vision_count %}
-                        {%- set image_count.value = image_count.value + 1 %}
-                    {%- endif %}
-                    {%- if add_vision_id %}
-                        {{- 'Picture ' ~ image_count.value ~ ': ' }}
-                    {%- endif %}
-                    {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
-                {%- elif item.type == 'video' or 'video' in item %}
-                    {%- if is_system_content %}
-                        {{- raise_exception('System message cannot contain videos.') }}
-                    {%- endif %}
-                    {%- if do_vision_count %}
-                        {%- set video_count.value = video_count.value + 1 %}
-                    {%- endif %}
-                    {%- if add_vision_id %}
-                        {{- 'Video ' ~ video_count.value ~ ': ' }}
-                    {%- endif %}
-                    {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
-                {%- elif 'text' in item %}
-                    {{- item.text }}
-                {%- else %}
-                    {{- raise_exception('Unexpected item type in content.') }}
-                {%- endif %}
-            {%- else %}
-                {{- item | string }}
-            {%- endif %}
-        {%- endfor %}
-    {%- elif content is none or content is undefined %}
-        {{- '' }}
-    {%- else %}
-        {{- raise_exception('Unexpected content type.') }}
-    {%- endif %}
-{%- endmacro %}
-{%- if not messages %}
-    {{- raise_exception('No messages provided.') }}
-{%- endif %}
-{%- set head = namespace(count=0, seen_non_system=false) %}
-{%- for message in messages %}
-    {%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
-    {%- if _is_sys and not head.seen_non_system %}
-        {%- set head.count = head.count + 1 %}
-    {%- else %}
-        {%- set head.seen_non_system = true %}
-    {%- endif %}
-{%- endfor %}
-{%- set sys_state = namespace(content='') %}
-{%- for message in messages[:head.count] %}
-    {%- set _part = render_content(message.content, false, true) | trim %}
-    {%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
-    {%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
-    {%- if _part %}
-        {%- if sys_state.content %}
-            {%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
-        {%- else %}
-            {%- set sys_state.content = _part %}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- set _sc = sys_state.content %}
-{%- set _msgs = messages[head.count:] %}
-{%- if ns_state.thinking %}
-    {%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
-{%- else %}
-    {%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}
-{%- endif %}
-{%- set _terse_core %}
-Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
-Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
-If a user request is genuinely ambiguous, ask a sharp question, don't guess.
-{%- endset %}
-{%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}
-{#- `terse` chat-template kwarg: default true, so behaviour is unchanged unless a caller opts out.
-    Pass {"terse": false} via chat_template_kwargs to serve the model with only its own system
-    prompt -- useful for A/B-ing the terseness effect, or when a downstream prompt conflicts. #}
-{%- set _terse_on = terse if terse is defined else true %}
-{%- if _terse_on %}
-    {%- if not _sc %}
-        {%- set _sc = _terse | trim %}
-    {%- else %}
-        {%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}
-    {%- endif %}
-{%- endif %}
-{%- if _has_tools %}
-    {{- '<|im_start|>system\n' }}
-    {%- if reasoning_instructions %}
-        {{- reasoning_instructions + '\n\n' }}
-    {%- endif %}
-    {{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
-    {%- for tool in tools %}
-        {{- '\n' }}
-        {{- tool | tojson }}
-    {%- endfor %}
-    {{- '\n</tools>' }}
-    {%- if _tool_format == 'json' %}
-        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
-    {%- else %}
-        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
-    {%- endif %}
-    {%- if _sc %}
-        {{- '\n\n' + _sc }}
-    {%- endif %}
-    {{- '<|im_end|>\n' }}
-{%- else %}
-    {%- if _sc %}
-        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
-    {%- elif reasoning_instructions %}
-        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
-    {%- endif %}
-{%- endif %}
-{%- set _last_idx = _msgs | length - 1 %}
-{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
-{%- for message in _msgs[::-1] %}
-    {%- set index = (_msgs | length - 1) - loop.index0 %}
-    {%- if ns.multi_step_tool and message.role == 'user' %}
-        {%- set _rc = render_content(message.content, false) | trim %}
-        {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
-            {%- set ns.multi_step_tool = false %}
-            {%- set ns.last_query_index = index %}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- if ns.multi_step_tool %}
-    {%- if _last_idx > 50 %}
-        {%- set ns.last_query_index = _last_idx %}
-    {%- else %}
-        {%- set ns.last_query_index = 0 %}
-    {%- endif %}
-{%- endif %}
-{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
-{%- for message in _msgs %}
-    {%- set is_system = (message.role == "system" or message.role == "developer") %}
-    {%- set content = render_content(message.content, true, is_system) | trim %}
-    {%- if is_system or message.role == 'user' %}
-        {%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
-        {%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
-    {%- endif %}
-    {%- if is_system %}
-        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
-    {%- elif message.role == 'user' %}
-        {%- set ns2.consecutive_failures = 0 %}
-        {{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
-    {%- elif message.role == 'assistant' %}
-        {%- set reasoning_content = '' %}
-        {%- set _explicit_reasoning = '' %}
-        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
-            {%- if message.reasoning_content is string %}
-                {%- set _explicit_reasoning = message.reasoning_content %}
-            {%- else %}
-                {%- set _explicit_reasoning = message.reasoning_content | string %}
-            {%- endif %}
-        {%- elif message.thinking is defined and message.thinking is not none %}
-            {%- if message.thinking is string %}
-                {%- set _explicit_reasoning = message.thinking %}
-            {%- else %}
-                {%- set _explicit_reasoning = message.thinking | string %}
-            {%- endif %}
-        {%- elif message.reasoning is defined and message.reasoning is not none %}
-            {%- if message.reasoning is string %}
-                {%- set _explicit_reasoning = message.reasoning %}
-            {%- else %}
-                {%- set _explicit_reasoning = message.reasoning | string %}
-            {%- endif %}
-        {%- endif %}
-        {%- if _explicit_reasoning %}
-            {%- set _lead_end = '' %}
-            {%- if content.startswith('<think>') and '</think>' in content %}
-                {%- set _lead_end = '</think>' %}
-            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
-                {%- set _lead_end = '</thinking>' %}
-            {%- elif content.startswith('</think>') %}
-                {%- set _lead_end = '</think>' %}
-            {%- elif content.startswith('</thinking>') %}
-                {%- set _lead_end = '</thinking>' %}
-            {%- endif %}
-            {%- if _lead_end %}
-                {%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
-            {%- endif %}
-            {%- set reasoning_content = _explicit_reasoning %}
-        {%- else %}
-            {%- set _think_end = '' %}
-            {%- if content.startswith('</think>') %}
-                {%- set _think_end = '</think>' %}
-            {%- elif content.startswith('</thinking>') %}
-                {%- set _think_end = '</thinking>' %}
-            {%- elif '\n</think>' in content %}
-                {%- set _think_end = '\n</think>' %}
-            {%- elif '\n</thinking>' in content %}
-                {%- set _think_end = '\n</thinking>' %}
-            {%- elif '\n</ think>' in content %}
-                {%- set _think_end = '\n</ think>' %}
-            {%- elif '\n</think >' in content %}
-                {%- set _think_end = '\n</think >' %}
-            {%- elif content.startswith('<think>') and '</think>' in content %}
-                {%- set _think_end = '</think>' %}
-            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
-                {%- set _think_end = '</thinking>' %}
-            {%- endif %}
-            {%- if _think_end %}
-                {%- if 'thinking' in _think_end %}
-                    {%- set _think_start = '<thinking>' %}
-                {%- else %}
-                    {%- set _think_start = '<think>' %}
-                {%- endif %}
-                {%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
-                {%- if _think_start in reasoning_content %}
-                    {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
-                {%- endif %}
-                {%- set content = content.split(_think_end)[-1].lstrip('\n') %}
-            {%- endif %}
-        {%- endif %}
-        {%- set reasoning_content = reasoning_content | trim %}
-        {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
-            {{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
-        {%- else %}
-            {{- '<|im_start|>assistant\n' + content }}
-        {%- endif %}
-        {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
-            {%- for tool_call in message.tool_calls %}
-                {%- if tool_call.function is defined and tool_call.function is not none %}
-                    {%- set tc = tool_call.function %}
-                {%- else %}
-                    {%- set tc = tool_call %}
-                {%- endif %}
-                {%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
-                {%- if _tool_format == 'json' %}
-                    {%- if loop.first %}
-                        {%- if content | trim %}
-                            {{- '\n\n' }}
-                        {%- endif %}
-                    {%- else %}
-                        {{- '\n' }}
-                    {%- endif %}
-                    {%- set _args = '{}' %}
-                    {%- if tc.arguments is defined and tc.arguments is not none %}
-                        {%- if tc.arguments is mapping %}
-                            {%- set _args = tc.arguments | tojson %}
-                        {%- elif tc.arguments is string %}
-                            {%- if tc.arguments %}
-                                {%- set _args = tc.arguments %}
-                            {%- endif %}
-                        {%- else %}
-                            {%- set _args = tc.arguments | tojson %}
-                        {%- endif %}
-                    {%- endif %}
-                    {{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
-                {%- else %}
-                    {%- if loop.first %}
-                        {%- if content | trim %}
-                            {{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
-                        {%- else %}
-                            {{- '<tool_call>\n<function=' + tc_name + '>\n' }}
-                        {%- endif %}
-                    {%- else %}
-                        {{- '\n<tool_call>\n<function=' + tc_name + '>\n' }}
-                    {%- endif %}
-                    {%- if tc.arguments is defined and tc.arguments is not none %}
-                        {%- if tc.arguments is mapping %}
-                            {%- for args_name, args_value in tc.arguments.items() %}
-                                {{- '<parameter=' + args_name + '>\n' }}
-                                {%- if args_value is string %}
-                                    {%- set _av = args_value %}
-                                {%- else %}
-                                    {%- set _av = args_value | tojson %}
-                                {%- endif %}
-                                {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
-                                    {{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
-                                {%- else %}
-                                    {{- _av }}
-                                {%- endif %}
-                                {{- '\n</parameter>\n' }}
-                            {%- endfor %}
-                        {%- else %}
-                            {%- if tc.arguments is string %}
-                                {%- set _raw_args = tc.arguments %}
-                            {%- else %}
-                                {%- set _raw_args = tc.arguments | tojson %}
-                            {%- endif %}
-                            {%- if _raw_args %}
-                                {%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
-                                    {{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
-                                {%- else %}
-                                    {{- _raw_args }}
-                                {%- endif %}
-                            {%- endif %}
-                        {%- endif %}
-                    {%- endif %}
-                    {{- '</function>\n</tool_call>' }}
-                {%- endif %}
-            {%- endfor %}
-        {%- endif %}
-        {{- '<|im_end|>\n' }}
-    {%- elif message.role == 'tool' %}
-        {%- set _content_lower = content | lower %}
-        {%- set _content_head = _content_lower[:120] %}
-        {%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
-        {%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
-        {%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
-        {%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
-        {%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
-        {%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
-        {%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
-            {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
-        {%- else %}
-            {%- set ns2.consecutive_failures = 0 %}
-        {%- endif %}
-        {%- if ns2.prev_role != 'tool' %}
-            {{- '<|im_start|>user' }}
-        {%- endif %}
-        {%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
-            {%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
-        {%- endif %}
-        {{- '\n<tool_response>\n' + content }}
-        {%- if ns2.consecutive_failures >= 2 %}
-            {{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
-        {%- elif ns2.consecutive_failures == 1 %}
-            {{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
-        {%- endif %}
-        {{- '\n</tool_response>' }}
-        {%- if loop.last %}
-            {{- '<|im_end|>\n' }}
-        {%- else %}
-            {%- set _next_role = _msgs[loop.index0 + 1].role %}
-            {%- if _next_role != 'tool' %}
-                {{- '<|im_end|>\n' }}
-            {%- endif %}
-        {%- endif %}
-    {%- else %}
-        {{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
-    {%- endif %}
-    {%- set ns2.prev_role = message.role %}
-{%- endfor %}
-{%- if add_generation_prompt %}
-    {{- '<|im_start|>assistant\n' }}
-    {%- if not ns_state.thinking %}
-        {{- '<think>\n\n</think>\n\n' }}
-    {%- else %}
-        {{- '<think>\n' }}
-    {%- endif %}
-{%- endif %}

scripts/.AGENTTRASH/download-q8agent.sh.rocmfpx

@@ -1,86 +0,0 @@
-#!/usr/bin/env bash
-# Download the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT payload for
-# frank-medium from HuggingFace into
-# /home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT.
-#
-# Run on Dankserver (the box that serves it).  The repo is public, so the
-# download works anonymously; a token in ~/.cache/huggingface/token is used
-# when present, which keeps the ~28 GB main file clear of the anonymous
-# rate limit at full speed.
-#
-#   bash scripts/download-q8agent.sh
-#   bash scripts/download-q8agent.sh --verify-only
-#
-# The script is resumable (curl -C -) and verifies every artifact's size
-# afterwards, so a truncated file fails loudly instead of load-crashing.
-set -euo pipefail
-
-REPO="kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF"
-DIR="/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT"
-EXPECTED=(
-  "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf:28214425952"
-  "mmproj-Qwen3.8-27B-BF16.gguf:931145888"
-  "mtp-Qwen3.8-27B-Q4_0.gguf:1680271648"
-)
-# The model card's MTP prompt-cache fix ships in the same repo as a patch;
-# fetched for reference only.  This fork's main already carries the
-# speculative boundary in the context checkpoint, so nothing applies it.
-EXTRA=(
-  "patches/mtp-prompt-cache-fix.patch"
-)
-
-cd -- "$(dirname -- "$0")/.."
-
-main() {
-  local mode="download"
-  [[ "${1:-}" == "--verify-only" ]] && mode="verify"
-  mkdir -p "$DIR"
-  cd "$DIR"
-
-  local token="" auth=()
-  if [[ -r "$HOME/.cache/huggingface/token" ]]; then
-    token="$(cat "$HOME/.cache/huggingface/token")"
-    auth=(-H "Authorization: Bearer $token")
-  else
-    echo "no HF token at $HOME/.cache/huggingface/token; downloading anonymously"
-  fi
-
-  dl() {
-    local f="$1"
-    mkdir -p "$(dirname "$f")"
-    curl -fsSL -C - -o "$f" --retry 10 --retry-delay 5 --retry-max-time 10800 \
-      "${auth[@]}" \
-      "https://huggingface.co/$REPO/resolve/main/$f"
-    echo "  done: $f ($(du -h "$f" | cut -f1))"
-  }
-
-  if [[ "$mode" == "download" ]]; then
-    echo "downloading $REPO into $DIR"
-    for entry in "${EXPECTED[@]}"; do
-      dl "${entry%%:*}"
-    done
-    for f in "${EXTRA[@]}"; do
-      dl "$f"
-    done
-  fi
-
-  echo "verifying sizes"
-  local ok=1 f want act
-  for entry in "${EXPECTED[@]}"; do
-    f="${entry%%:*}"; want="${entry##*:}"
-    act="$(stat -c%s "$f" 2>/dev/null || echo 0)"
-    if [[ "$act" -eq "$want" ]]; then
-      echo "  OK   $f ($act bytes)"
-    else
-      echo "  BAD  $f ($act bytes, want $want)" >&2
-      ok=0
-    fi
-  done
-  if [[ -z "$token" ]]; then
-    echo "note: no token was used; sizes above are the integrity check"
-  fi
-  [[ "$ok" -eq 1 ]] || exit 1
-  echo "all files present and sized correctly"
-}
-
-main "$@"

scripts/.AGENTTRASH/frank-default-npu.env

@@ -1,17 +0,0 @@
-# Tunables for frank-default-npu (FastFlowLM NPU endpoint, Dankclaw)
-# Runtime: FLM 1.0.1 on the XDNA2 NPU (50 TOPS). Model: Gemma4-E4B-IT
-# (8B, Q4_1, vision + audio + reasoning). The NPU allocates from the
-# unified pool shared with frank-default-large (the CIRU llama.cpp
-# service on the iGPU). See serverAppConfigs-docs/FastFlowLM NPU Inference.md.
-#
-# 2026-08-20: gemma4-it:e4b deployed. Footprint ~9.1 GB. 64k context.
-
-FLM_MODEL=gemma4-it:e4b
-FLM_HOST=0.0.0.0
-FLM_PORT=52626
-FLM_PMODE=performance
-FLM_ASR=0
-# 64K context (native max for gemma4-it is 32768, but NPU supports up to 256k).
-# The model's max_prefill_len is 4096, so prompts longer than that get
-# truncated at prefill time.
-FLM_CTX_LEN=65536

scripts/.AGENTTRASH/frank-default-npu.service

@@ -1,17 +0,0 @@
-[Unit]
-Description=FastFlowLM NPU inference (frank-default-npu) on Dankclaw
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-Environment="FLM_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-npu.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-default-npu.sh
-LimitMEMLOCK=infinity
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

scripts/.AGENTTRASH/frank-dragoon.env.ik-llama.cpp

@@ -1,105 +0,0 @@
-# Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
-# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
-# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
-#
-# Runtime: ik_llama.cpp master d85a1d7 (2026-09-15) with full qwen4exp
-# support (upstream PRs #2365/#2369/#2373 merged). Build recipe (CUDA for
-# the 3090 at sm_86, Zen5 AVX-512 IQK kernels for the CPU expert path):
-#   cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON \
-#     -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 \
-#     -DGGML_AVX512=ON -DGGML_AVX512_VBMI=ON -DGGML_AVX512_VNNI=ON \
-#     -DGGML_AVX512_BF16=ON -DLLAMA_CURL=OFF
-# The startup banner must print HAVE_FANCY_SIMD is defined — without those
-# macros the quantized GEMM silently falls back to AVX2 (no warning).
-#
-# Model: mradermacher imatrix quant of the uncensored Flash-Next (qwen4exp
-# MoE: 512 experts / 10 used, 48 layers = 12 attention + 36 GDN/ssm, QSA
-# indexer, hyper-connections, PLE n-gram table ~320M rows). 111 GiB shipped
-# as 3 parts, concatenated on disk; each part's sha256 was verified against
-# its HF LFS oid at download time.
-#
-# Placement (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
-#   - -ot "exps=CPU": ALL 512-expert tensors (121B elems, ~84 GB) live in
-#     the Strix Halo's LPDDR5X via the plain CPU backend buffer — mmap-backed
-#     page cache, ik's fastest known configuration for this shape (the
-#     author's 2x3090 + Threadripper reference box measures ~50 t/s on the
-#     same model family). The shared experts (shexp) keep their GPU-side
-#     default: they run on every token.
-#   - -ot "per_layer_token_embd=CPU": the ~32 GB PLE n-gram table follows —
-#     only a tiny slice is touched per token, so it streams from page cache.
-#   - NEVER use --cpu-moe on this host: it routes the CPU-overridden
-#     weights through the CUDA_Host PINNED buffer (measured 2026-09-15:
-#     108.12 GiB page-locked, evicted every co-tenant into swap and starved
-#     sshd into unreachability until a power cycle). -ot ...=CPU is the
-#     mmap-backed path with identical DDR5 bandwidth for resident pages,
-#     and the kernel degrades gracefully under pressure (page cache evict
-#     + re-read) instead of stranding anyone.
-#   - GGML_CUDA_NO_PINNED_WEIGHTS=1 (src/llama-load-tensors.cpp) keeps the
-#     CPU-overridden weights mmapped and leaves only the small staging
-#     buffers pinned. (GGML_CUDA_NO_PINNED would also unpin staging —
-#     unnecessary here and slower for prefill transfers.)
-#
-# Sampling: greedy-by-default to preserve the halogen-era client contract
-# (a request without temperature decodes greedy; temp 0.0 = argmax).
-# Clients that send their own temperature get their own values.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (see
-# the 2026-08-13 HIP incident in AGENTS.md). llama.cpp reads --host and
-# --temp as CLI flags, never as environment variables.
-
-# ik_llama.cpp runtime binary (master d85a1d7 build).
-LLAMA_BIN=/home/dank/GitPackages/ik_llama.cpp/build/bin/llama-server
-
-# Model package from
-# https://huggingface.co/mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF
-# (parts concatenated). The mmproj is the matching vision tower from the
-# static repo mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF — ik_llama.cpp
-# links examples/mtmd, so image input survives the runtime swap.
-MODEL_DIR=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M
-MODEL_FILE=Qwen3.8-Flash-Next-Uncensored.i1-Q4_K_M.gguf
-MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M/Qwen3.8-Flash-Next-Uncensored.mmproj-f16.gguf
-
-# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
-# server emits reasoning in DeepSeek format so clients receive
-# reasoning_content separately instead of literal think tags in content.
-CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
-
-# Alias + network.
-ALIAS=frank-dragoon
-BIND_ADDR=0.0.0.0
-LISTEN_PORT=7770
-
-# Serving: single 262144-token slot (the model's full context window),
-# batch 4096 / ubatch 512, 16+16 threads (the Halo CPU has 16 cores, no
-# SMT), f16 KV. The KV cache scales with the 12 full-attention layers only
-# (~6 GB f16 at 262k); the GDN/ssm layers keep constant-size state. If
-# VRAM headroom is needed, --cache-type-k/v q8_0 halves the KV footprint.
-CTX_SIZE=262144
-PARALLEL=1
-BATCH_SIZE=4096
-UBATCH_SIZE=512
-NUM_THREADS=16
-THREAD_BATCH=16
-CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
-
-# GPU/MoE placement: everything on the 3090 except the experts and the PLE
-# table, which both live in host RAM via the CPU backend buffer (see the
-# placement notes above).
-NGL=999
-# ik_llama.cpp CUDA host-memory policy: CPU-overridden weights stay mmapped;
-# only the small staging buffers are pinned (see the placement notes above).
-GGML_CUDA_NO_PINNED_WEIGHTS=1
-
-# Prompt cache (RAM budget in MiB) and slot state dir.
-CACHE_RAM=8192
-SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
-
-# Greedy-by-default (halogen contract). See sampling note above.
-TEMPERATURE=0.0
-
-# Reasoning on: DeepSeek format projects thinking into the API's
-# reasoning_content field instead of literal think tags in content;
-# unlimited reasoning budget (the model decides).
-REASONING_FORMAT=deepseek
-REASONING_BUDGET=-1

scripts/.AGENTTRASH/frank-dragoon.service.ik-llama.cpp

@@ -1,21 +0,0 @@
-[Unit]
-Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-# Staging buffers for the CUDA transfers are page-locked; the weights
-# themselves stay mmapped (GGML_CUDA_NO_PINNED_WEIGHTS=1), but the staging
-# budget still needs room. Parity with the halogen quadlet's
-# Ulimit=memlock=-1:-1.
-LimitMEMLOCK=infinity
-Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

scripts/.AGENTTRASH/frank-medium.env.rocmfpx

@@ -1,138 +0,0 @@
-# Tunables for frank-medium (kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF,
-# charlie12345/ROCmFPX llama.cpp, Dankserver, 2x AMD Radeon AI PRO R9700).
-#
-# Hardware (2026-09-15): the RTX 3090 left Dankserver and two Radeon AI PRO
-# R9700 cards (Navi 48, gfx1201, RDNA4, 32,624 MiB each, PCIe 5.0 x16) took
-# its place.  The Raphael iGPU (gfx1036) is still enumerated by HIP as the
-# third device.  There is no XGMI link between the two cards and they sit in
-# different IOMMU groups, so any cross-GPU traffic goes over PCIe.
-#
-# Runtime: charlie12345/ROCmFPX (llama.cpp fork, ROCmFPX quant family; the
-# Q8_0_ROCMFPX_AGENT tensor type exists nowhere else - stock llama.cpp cannot
-# load this GGUF).  Built for gfx1201 with the fork's self-contained ROCm
-# 7.14 toolchain so the host's stale /opt/rocm-6.3.3 install is never used:
-#
-#   cd /home/dank/GitPackages/charlie-ROCmFPX
-#   env JOBS=8 ./scripts/build-rocmfp4-rocm714-local.sh
-#
-# gfx1201 must be the build target: a gfx1200 binary links, loads the model
-# and then segfaults on this card (fork's docs/BUILD-AMD-ARCHITECTURES.md).
-#
-# Model: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT - Q8_0_ROCMFPX_AGENT
-# (ftype 115), 26.28 GiB, 8.39 bpw, arch qwen35 (MoE), quantized from a
-# 51.3 GiB BF16 source.  The AGENT routing keeps 12 more attention-output
-# projections at full Q8_0 than the plain 8-bit build, which buys a higher
-# MTP draft acceptance (0.953 vs 0.911) and that is the whole point of this
-# checkpoint.  Model card documents 131,072 context.
-#
-# Parallelism (--split-mode layer, operator call 2026-09-15 "pipeline
-# parallelism"): layers are distributed across the two cards in order and
-# activations hop PCIe at the boundary.  Measured with llama-bench
-# (pp4096 / tg128 / same at depth 32768, f16 KV, fa on, 3 reps):
-#
-#   single R9700   1160.9 / 19.42   (fits: 26.27 GiB weights, ~5 GiB left)
-#   layer  2x      1991.2 / 18.76   859.1 / 17.64
-#   tensor 2x       271.8 / 26.26   228.6 / 25.09
-#
-# Tensor parallel wins decode (+40 %) but collapses prefill 7x: this build
-# has no RCCL (llama.cpp logs "AllReduce init failed; falling back to
-# meta-backend butterfly"), so every layer pays a slow cross-card reduction.
-# Agentic traffic is prefill-heavy, so layer split wins the workload.  Revisit
-# if the fork ships a working RCCL tensor path.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
-# ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
-# incident).  llama.cpp reads --host/--port/--temp as CLI flags, never as
-# env vars.
-
-# charlie12345/ROCmFPX llama-server (self-contained ROCm 7.14 build, RPATH
-# points at the bundled runtime libs; no LD_LIBRARY_PATH needed).
-LLAMA_BIN=/home/dank/GitPackages/charlie-ROCmFPX/build-rdna4-rocm714/bin/llama-server
-
-# Model package, fetched with scripts/download-q8agent.sh:
-#   Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf   26.28 GiB  main weights
-#   mtp-Qwen3.8-27B-Q4_0.gguf              1.68 GiB  MTP draft head
-#   mmproj-Qwen3.8-27B-BF16.gguf           0.93 GiB  vision tower (unused)
-MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
-MODEL_FILE=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
-DRAFT_MODEL_FILE=mtp-Qwen3.8-27B-Q4_0.gguf
-
-# Alias + network.  The port is the fleet's dankserver endpoint; the alias is
-# what clients ask for, and it replaces the retired `frank-small`.
-ALIAS=frank-medium
-BIND_ADDR=0.0.0.0
-LISTEN_PORT=7770
-
-# Device placement.  HIP_VISIBLE_DEVICES hides the iGPU (device 2) so nothing
-# can drift onto shared system RAM; DEVICES is what llama.cpp splits across.
-HIP_VISIBLE_DEVICES=0,1
-DEVICES=ROCm0,ROCm1
-SPLIT_MODE=layer
-
-# TENSOR_SPLIT empty = layers distributed by free VRAM (equal here).
-TENSOR_SPLIT=
-
-NGL=999
-# Auto-fit has no business resizing a context we set explicitly.
-FIT=off
-FLASH_ATTN=on
-
-# 131,072 tokens, one slot (the model card's documented window; the GGUF
-# reports n_ctx_train 262144, untested by the model author).  Layer split
-# keeps each layer's KV on the card that owns the layer, so the window is
-# bounded by per-card free memory: ~5.6 GiB of KV per card at 131072 f16.
-CTX_SIZE=131072
-PARALLEL=1
-
-# 26.28 GiB of weights + MTP draft + KV cache leaves ~15 GiB per card at this
-# context, so the batch shape is chosen for prefill throughput, not memory.
-BATCH_SIZE=2048
-UBATCH_SIZE=512
-NUM_THREADS=8
-THREAD_BATCH=8
-
-# f16 K/V: an agent checkpoint is the wrong place to spend KV precision.
-CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
-CTX_CHECKPOINTS=8
-CACHE_RAM=8192
-
-# Sampling: the checkpoint's instruct table.
-TEMPERATURE=0.7
-TOP_P=0.8
-TOP_K=20
-MIN_P=0.0
-PRESENCE_PENALTY=1.5
-
-# Reasoning on; DeepSeek format projects the trace into reasoning_content
-# instead of literal thinking tags in content.
-REASONING=on
-REASONING_FORMAT=deepseek
-REASONING_BUDGET=-1
-
-# Chat template: empty = the template embedded in the GGUF.  Point this at
-# /home/dank/serverAppConfigs/scripts/chat-template.jinja to force the repo's
-# Sharp agent template instead.
-CHAT_TEMPLATE_FILE=
-CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
-
-# MTP speculative decoding.  n_max 6 / p_min 0.6 is the fork's measured
-# sweet spot for a MTP head with high acceptance; p_min 0.0 drafts
-# low-probability tokens that get rejected and makes MTP a net loss.
-SPEC_TYPE=draft-mtp
-SPEC_DRAFT_N_MAX=6
-SPEC_DRAFT_N_MIN=0
-SPEC_DRAFT_P_MIN=0.6
-SPEC_DRAFT_P_SPLIT=0.10
-SPEC_DRAFT_NGL=999
-SPEC_DRAFT_TYPE_K=f16
-SPEC_DRAFT_TYPE_V=f16
-SPEC_DRAFT_POLL=1
-SPEC_DRAFT_POLL_BATCH=1
-# The draft head rides the card that holds the target model's LAST layers:
-# the MTP head consumes the final hidden state, so pinning it to ROCm1 keeps
-# the per-token hop off the wire.  (Under the abandoned tensor split the
-# draft must also be pinned to exactly one device or the meta-device KV
-# allocation aborts: GGML_ASSERT(buffer) in ggml-backend.cpp, measured
-# 2026-09-15.)
-SPEC_DRAFT_DEVICE=ROCm1

scripts/.AGENTTRASH/frank-medium.service.rocmfpx

@@ -1,16 +0,0 @@
-[Unit]
-Description=charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT qwen35 MoE (dankserver, frank-medium, 2x Radeon AI PRO R9700)
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User=dank
-Group=dank
-Environment="FRANK_MEDIUM_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-medium.env"
-ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-medium.sh
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

scripts/.AGENTTRASH/serve-frank-default-npu.sh

@@ -1,49 +0,0 @@
-#!/usr/bin/env bash
-# Serve the FastFlowLM (FLM) NPU endpoint — frank-default-npu.
-#
-# Runs flm serve with the model tag in frank-default-npu.env. The NPU
-# (XDNA2, 50 TOPS) is a single-lane device: generations serialize in FIFO
-# order, so this is a background/triage endpoint. The NPU shares the ~30 GiB
-# unified pool with frank-default-large (the CIRU llama.cpp service on the
-# iGPU); see serverAppConfigs-docs/FastFlowLM NPU Inference.md for the memory budget.
-#
-# NOTE: FLM requires unlimited memlock. The unit sets LimitMEMLOCK=infinity.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FLM_NPU_ENV_FILE:-$SCRIPT_DIR/frank-default-npu.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -------------------
-: "${FLM_BIN:=/usr/bin/flm}"
-: "${FLM_MODEL:=gemma4-it:e4b}"
-: "${FLM_HOST:=0.0.0.0}"
-: "${FLM_PORT:=52626}"
-: "${FLM_PMODE:=performance}"
-: "${FLM_ASR:=0}"
-: "${FLM_CTX_LEN:=-1}"
-
-if [[ ! -x "$FLM_BIN" ]]; then
-    echo "missing flm binary: $FLM_BIN" >&2
-    exit 1
-fi
-
-# --- optional flags ----------------------------------------------------------
-OPT_ARGS=()
-if [[ "${FLM_ASR}" != "0" ]]; then
-    OPT_ARGS+=(-a "${FLM_ASR}")
-fi
-if [[ "${FLM_CTX_LEN}" != "-1" ]]; then
-    OPT_ARGS+=(-c "${FLM_CTX_LEN}")
-fi
-
-exec "$FLM_BIN" serve "$FLM_MODEL" \
-    --host "$FLM_HOST" \
-    -p "$FLM_PORT" \
-    --pmode "$FLM_PMODE" \
-    "${OPT_ARGS[@]}"

scripts/.AGENTTRASH/serve-frank-dragoon.sh.ik-llama.cpp

@@ -1,123 +0,0 @@
-#!/usr/bin/env bash
-# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-dragoon
-# via ik_llama.cpp on Dankclaw.
-#
-# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
-# the RTX 3090 (CUDA, sm_86) takes the dense/attention/output weights and
-# the KV cache (-ngl 999); ALL MoE expert tensors and the ~32 GB PLE
-# n-gram table stay in the Strix Halo's LPDDR5X via the plain CPU backend
-# buffer (-ot ...=CPU — mmap-backed page cache; only a tiny slice of PLE is
-# touched per token). NEVER use --cpu-moe on this host: it routes the
-# CPU-overridden weights through the CUDA_Host PINNED buffer (measured
-# 108.12 GiB page-locked on 2026-09-15, which evicted every co-tenant into
-# swap and starved sshd into unreachability until a power cycle). NEVER add
-# --no-mmap — the STRIX-era lesson transfers directly (the cgroup killer
-# takes the process with nothing in the server log).
-#
-# Runtime: ik_llama.cpp master d85a1d7 built with CUDA (sm_86) + Zen5
-# AVX-512 IQK kernels; the startup banner must show HAVE_FANCY_SIMD
-# is defined. Flag set verified against this binary's --help before this
-# script was written. ik has no --cache-prompt/--cache-idle-slots (prompt
-# caching is its default behavior) and no --alias: it is a single-model
-# server and reports the GGUF's general.name at /v1/models.
-#
-# Sampling: greedy-by-default (halogen client contract) — clients that
-# send their own temperature get their own values.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -------------------
-: "${LLAMA_BIN:=/home/dank/GitPackages/ik_llama.cpp/build/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M}"
-: "${MODEL_FILE:=Qwen3.8-Flash-Next-Uncensored.i1-Q4_K_M.gguf}"
-: "${MMPROJ_PATH:=$MODEL_DIR/Qwen3.8-Flash-Next-Uncensored.mmproj-f16.gguf}"
-: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
-: "${BIND_ADDR:=0.0.0.0}"
-: "${LISTEN_PORT:=7770}"
-: "${CTX_SIZE:=262144}"
-: "${PARALLEL:=1}"
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=512}"
-: "${NUM_THREADS:=16}"
-: "${THREAD_BATCH:=16}"
-: "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-: "${NGL:=999}"
-: "${CPU_MOE:=on}"
-: "${FLASH_ATTN:=on}"
-: "${TEMPERATURE:=0.0}"
-: "${REASONING:=on}"
-: "${REASONING_FORMAT:=deepseek}"
-: "${REASONING_BUDGET:=-1}"
-: "${CACHE_RAM:=8192}"
-: "${SLOT_SAVE_PATH:=$SCRIPT_DIR/slot-state}"
-# Hard guard: on this host CUDA pinned host memory strands the co-tenants
-# (measured 2026-09-15: a default-policy start tried to page-lock 108.12 GiB,
-# evicted every co-tenant into swap and starved sshd into unreachability
-# until a power cycle). GGML_CUDA_NO_PINNED_WEIGHTS=1 keeps the
-# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
-# sets it. Refuse to run without it.
-if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
-    echo "frank-dragoon: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
-    exit 1
-fi
-
-mkdir -p "$SLOT_SAVE_PATH"
-
-# The UVM device node is created lazily by nvidia-modprobe on the first
-# CUDA open after boot; make sure it exists before the server needs it.
-nvidia-modprobe -u -c=0 2>/dev/null || true
-
-CPU_MOE_ARGS=()
-if [[ "$CPU_MOE" == "on" ]]; then
-    # -ot ...=CPU (plain CPU backend buffer) is the mmap-backed path. NEVER
-    # swap this for --cpu-moe: that routes the weights through the CUDA_Host
-    # PINNED buffer (108.12 GiB measured) and strands the co-tenants.
-    CPU_MOE_ARGS=(-ot "exps=CPU" -ot "per_layer_token_embd=CPU")
-fi
-
-MMPROJ_ARGS=()
-if [[ -n "${MMPROJ_PATH:-}" && -f "$MMPROJ_PATH" ]]; then
-    MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
-fi
-
-TEMPLATE_ARGS=()
-if [[ -n "${CHAT_TEMPLATE_FILE:-}" && -f "$CHAT_TEMPLATE_FILE" ]]; then
-    TEMPLATE_ARGS=(--chat-template-file "$CHAT_TEMPLATE_FILE")
-else
-    TEMPLATE_ARGS=(--jinja)
-fi
-
-exec "$LLAMA_BIN" \
-    --model "$MODEL_DIR/$MODEL_FILE" \
-    "${TEMPLATE_ARGS[@]}" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --gpu-layers "$NGL" \
-    "${CPU_MOE_ARGS[@]}" \
-    --flash-attn "$FLASH_ATTN" \
-    --parallel "$PARALLEL" \
-    --ctx-size "$CTX_SIZE" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --threads "$NUM_THREADS" \
-    --threads-batch "$THREAD_BATCH" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --cache-ram "$CACHE_RAM" \
-    --slot-save-path "$SLOT_SAVE_PATH" \
-    --ctx-checkpoints 16 \
-    --metrics \
-    --temp "$TEMPERATURE" \
-    --reasoning "$REASONING" \
-    --reasoning-format "$REASONING_FORMAT" \
-    --reasoning-budget "$REASONING_BUDGET" \
-    "${MMPROJ_ARGS[@]}"

scripts/.AGENTTRASH/serve-frank-medium.sh.rocmfpx

@@ -1,215 +0,0 @@
-#!/usr/bin/env bash
-# Serve kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium via
-# the charlie12345/ROCmFPX llama.cpp runtime on Dankserver
-# (2x AMD Radeon AI PRO R9700, Navi 48, gfx1201, 32 GB each).
-#
-# Runtime: charlie12345/ROCmFPX main (c49ebdb, 2026-08-22), built for
-# gfx1201 with the fork's self-contained ROCm toolchain:
-#
-#   cd /home/dank/GitPackages/charlie-ROCmFPX
-#   env JOBS=8 ./scripts/build-rocmfp4-rocm714-local.sh
-#   -> build-rdna4-rocm714/bin/llama-server
-#
-# That script downloads the gfx120X-all TheRock ROCm 7.14 toolchain into
-# ~/.local/share/rocm-7.14.0a20260624, builds with
-# -DCMAKE_HIP_ARCHITECTURES="gfx1200;gfx1201", copies the ROCm runtime
-# (hip/hsa/rocblas/hipblaslt + their kernel libraries) next to the binary and
-# sets RPATH=$ORIGIN/../lib on it.  The resulting service is self-contained:
-# no LD_LIBRARY_PATH, and the host's stale /opt/rocm-6.3.3 cannot leak in.
-# gfx1201 is NOT interchangeable with gfx1200 on this card (the fork's
-# docs/BUILD-AMD-ARCHITECTURES.md: a gfx1200 build links, loads and then
-# segfaults), and the build verifies the emitted code objects afterwards.
-#
-# Model: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT (Q8_0_ROCMFPX_AGENT,
-# ftype 115, arch qwen35 MoE), 26.28 GiB main file, plus a separate
-# MTP draft head and an mmproj vision tower.  8.39 bpw; the AGENT routing
-# keeps 13 of the 17 attention-output projections at full Q8_0, which is
-# what makes the MTP draft land ~0.95 of its guesses.  A stock llama.cpp
-# cannot load this GGUF at all - Q8_0_ROCMFPX exists only in this fork.
-#
-# Parallelism: --split-mode layer across both cards (operator call
-# 2026-09-15).  Layers are distributed in order and activations hop PCIe at
-# the boundary; measured llama-bench: layer 1991/18.8 tok/s (pp4096/tg128)
-# vs tensor 272/26.3 and single-card 1161/19.4.  Tensor wins decode +40 %
-# but its prefill collapses 7x on this build (no RCCL: "AllReduce init
-# failed; falling back to meta-backend butterfly"), so layer split wins the
-# prefill-heavy agentic workload.
-#
-# Tunables come from frank-medium.env unless FRANK_MEDIUM_ENV_FILE
-# overrides that path.
-set -euo pipefail
-
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_MEDIUM_ENV_FILE:-$SCRIPT_DIR/frank-medium.env}"
-if [[ -f "$ENV_FILE" ]]; then
-    set -a
-    # shellcheck disable=SC1090
-    source "$ENV_FILE"
-    set +a
-fi
-
-# --- defaults (used when the env file does not set them) -----------------
-: "${LLAMA_BIN:=/home/dank/GitPackages/charlie-ROCmFPX/build-rdna4-rocm714/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT}"
-: "${MODEL_FILE:=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
-: "${DRAFT_MODEL_FILE:=mtp-Qwen3.8-27B-Q4_0.gguf}"
-: "${ALIAS:=frank-medium}"
-: "${BIND_ADDR:=0.0.0.0}"
-: "${LISTEN_PORT:=7770}"
-
-# Devices: ROCm0/ROCm1 are the two R9700s; the Raphael iGPU enumerates as
-# ROCm2 (gfx1036) and is hidden from HIP so nothing can drift onto it.
-: "${HIP_VISIBLE_DEVICES:=0,1}"
-: "${DEVICES:=ROCm0,ROCm1}"
-: "${SPLIT_MODE:=tensor}"
-: "${TENSOR_SPLIT:=}"
-: "${NGL:=999}"
-: "${FIT:=off}"
-: "${FLASH_ATTN:=on}"
-
-: "${CTX_SIZE:=131072}"
-: "${PARALLEL:=1}"
-: "${BATCH_SIZE:=2048}"
-: "${UBATCH_SIZE:=512}"
-: "${NUM_THREADS:=8}"
-: "${THREAD_BATCH:=8}"
-: "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-: "${CTX_CHECKPOINTS:=8}"
-: "${CACHE_RAM:=8192}"
-
-# Sampling: the checkpoint's instruct table.
-: "${TEMPERATURE:=0.7}"
-: "${TOP_P:=0.8}"
-: "${TOP_K:=20}"
-: "${MIN_P:=0.0}"
-: "${PRESENCE_PENALTY:=1.5}"
-
-# Reasoning: the model is an agent checkpoint; keep the trace and project it
-# into the API's reasoning_content field rather than literal thinking tags.
-: "${REASONING:=on}"
-: "${REASONING_FORMAT:=deepseek}"
-: "${REASONING_BUDGET:=-1}"
-
-# Chat template: empty means "use the template embedded in the GGUF".
-: "${CHAT_TEMPLATE_FILE:=}"
-: "${CHAT_TEMPLATE_KWARGS:={\"enable_thinking\":true,\"preserve_thinking\":true}}"
-
-# MTP draft head (separate GGUF, --spec-type draft-mtp).  SPEC_DRAFT_DEVICE
-# must name exactly one device: under tensor mode an unsplit draft joins the
-# meta-device KV allocation and aborts while building the KV cache
-# (GGML_ASSERT(buffer) in ggml-backend.cpp, 2026-09-15).
-: "${SPEC_TYPE:=draft-mtp}"
-: "${SPEC_DRAFT_N_MAX:=6}"
-: "${SPEC_DRAFT_N_MIN:=0}"
-: "${SPEC_DRAFT_P_MIN:=0.6}"
-: "${SPEC_DRAFT_P_SPLIT:=0.10}"
-: "${SPEC_DRAFT_NGL:=999}"
-: "${SPEC_DRAFT_DEVICE:=}"
-: "${SPEC_DRAFT_TYPE_K:=f16}"
-: "${SPEC_DRAFT_TYPE_V:=f16}"
-: "${SPEC_DRAFT_POLL:=1}"
-: "${SPEC_DRAFT_POLL_BATCH:=1}"
-
-# Text only: the server must not try to autoload the vision tower.  Vision
-# stays on frank-dragoon (Dankclaw), which is where the fleet routes images.
-NO_MMPROJ="${NO_MMPROJ:-1}"
-
-export HIP_VISIBLE_DEVICES
-
-MODEL="$MODEL_DIR/$MODEL_FILE"
-if [[ ! -x "$LLAMA_BIN" ]]; then
-    echo "missing executable runtime: $LLAMA_BIN" >&2
-    echo "build it with: cd $(dirname "$(dirname "$LLAMA_BIN")")/.. && env JOBS=8 ./scripts/build-rocmfp4-rocm714-local.sh" >&2
-    exit 1
-fi
-if [[ ! -r "$MODEL" ]]; then
-    echo "missing or unreadable required artifact: $MODEL" >&2
-    echo "fetch it with: bash $SCRIPT_DIR/download-q8agent.sh" >&2
-    exit 1
-fi
-if [[ -n "$CHAT_TEMPLATE_FILE" && ! -r "$CHAT_TEMPLATE_FILE" ]]; then
-    echo "missing or unreadable chat template: $CHAT_TEMPLATE_FILE" >&2
-    exit 1
-fi
-
-# --- speculative decoding (MTP draft head) --------------------------------
-SPEC_ARGS=()
-if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
-    SPEC_ARGS=(
-        --spec-type "$SPEC_TYPE"
-        --spec-draft-n-max "$SPEC_DRAFT_N_MAX"
-        --spec-draft-n-min "$SPEC_DRAFT_N_MIN"
-        --spec-draft-p-min "$SPEC_DRAFT_P_MIN"
-        --spec-draft-p-split "$SPEC_DRAFT_P_SPLIT"
-    )
-    if [[ "$SPEC_TYPE" == *draft-mtp* ]]; then
-        DRAFT_MODEL="$MODEL_DIR/$DRAFT_MODEL_FILE"
-        if [[ ! -r "$DRAFT_MODEL" ]]; then
-            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
-            exit 1
-        fi
-        SPEC_ARGS+=(
-            --spec-draft-model "$DRAFT_MODEL"
-            --spec-draft-ngl "$SPEC_DRAFT_NGL"
-            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
-            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
-            --spec-draft-poll "$SPEC_DRAFT_POLL"
-            --spec-draft-poll-batch "$SPEC_DRAFT_POLL_BATCH"
-        )
-        if [[ -n "$SPEC_DRAFT_DEVICE" ]]; then
-            SPEC_ARGS+=(--spec-draft-device "$SPEC_DRAFT_DEVICE")
-        fi
-    fi
-fi
-
-# --- template + placement -------------------------------------------------
-TEMPLATE_ARGS=()
-if [[ -n "$CHAT_TEMPLATE_FILE" ]]; then
-    TEMPLATE_ARGS+=(--chat-template-file "$CHAT_TEMPLATE_FILE")
-    TEMPLATE_ARGS+=(--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS")
-fi
-
-PLACEMENT_ARGS=(--device "$DEVICES" --split-mode "$SPLIT_MODE")
-if [[ -n "$TENSOR_SPLIT" ]]; then
-    PLACEMENT_ARGS+=(--tensor-split "$TENSOR_SPLIT")
-fi
-
-MMPROJ_ARGS=()
-if [[ "$NO_MMPROJ" == "1" ]]; then
-    MMPROJ_ARGS=(--no-mmproj)
-fi
-
-exec "$LLAMA_BIN" \
-    --model "$MODEL" \
-    --alias "$ALIAS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --jinja \
-    --gpu-layers "$NGL" \
-    --fit "$FIT" \
-    --parallel "$PARALLEL" \
-    --ctx-size "$CTX_SIZE" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --flash-attn "$FLASH_ATTN" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --threads "$NUM_THREADS" \
-    --threads-batch "$THREAD_BATCH" \
-    --ctx-checkpoints "$CTX_CHECKPOINTS" \
-    --cache-ram "$CACHE_RAM" \
-    --cache-prompt \
-    --metrics \
-    --temp "$TEMPERATURE" \
-    --top-p "$TOP_P" \
-    --top-k "$TOP_K" \
-    --min-p "$MIN_P" \
-    --presence-penalty "$PRESENCE_PENALTY" \
-    --reasoning "$REASONING" \
-    --reasoning-format "$REASONING_FORMAT" \
-    --reasoning-budget "$REASONING_BUDGET" \
-    "${TEMPLATE_ARGS[@]}" \
-    "${PLACEMENT_ARGS[@]}" \
-    "${MMPROJ_ARGS[@]}" \
-    "${SPEC_ARGS[@]}"

scripts/abs-post-reorg.py

@@ -0,0 +1,204 @@
+#!/usr/bin/env python3
+"""abs-post-reorg.py — drive the Audiobookshelf side of the 2026-11-04 relayout.
+
+Runs ON Dankserver (the ABS host). Credentials come from the machine-local
+file ~/.config/secrets/audiobookshelf and never enter argv; requests go
+through urllib so neither the password nor the bearer token is world-readable
+in /proc. The delete arm is the DB-row arm ONLY — `hard=1` is fs.remove() of
+the item folder and must never reach this file (see the Audiobookshelf vault
+page; after the move the covering paths are whole books).
+
+Subcommands
+  covers  --list LIST.tsv        copy each ABS /metadata cover onto the book
+                                 folder ("srchost<TAB>dstarray" per line);
+                                 run BEFORE the mover so the cover rides the
+                                 book's move, or AFTER with dst already final
+  wipe    [--apply]              delete every library item DB row (progress
+                                 included — the campaign accepted losing the
+                                 11 progress rows); refuses without --apply
+  scan    [--wait]               trigger the library scan; --wait polls
+                                 isFetchingLibrary then prints the item total
+  status                         item total + isMissing census
+
+Paging law (measured twice): page on `page` (0-based) + `limit`; `offset` is
+ignored outright. Count unique ids; stop on `total`.
+"""
+import argparse
+import json
+import os
+import shutil
+import sys
+import time
+import urllib.error
+import urllib.request
+
+BASE = "http://127.0.0.1:8000/audiobookshelf"
+SECRET = os.environ.get("ABS_SECRET", "/home/dank/.config/secrets/audiobookshelf")
+
+
+def cred():
+    kv = {}
+    with open(SECRET) as fh:
+        for line in fh:
+            if "=" in line:
+                k, _, v = line.strip().partition("=")
+                kv[k] = v
+    return kv.get("ABS_ADMIN_USER", "frank"), kv.get("ABS_ADMIN_PASSWORD")
+
+
+def call(method, path, body=None, token=None, base=BASE):
+    data = json.dumps(body).encode() if body is not None else None
+    req = urllib.request.Request(base + path, data=data, method=method)
+    req.add_header("Content-Type", "application/json")
+    if token:
+        req.add_header("Authorization", f"Bearer {token}")
+    try:
+        with urllib.request.urlopen(req, timeout=30) as r:
+            raw = r.read().decode()
+            try:
+                return r.status, json.loads(raw)
+            except json.JSONDecodeError:
+                return r.status, raw.strip()
+    except urllib.error.HTTPError as e:
+        return e.code, e.read().decode()[:200]
+
+
+def login():
+    user, password = cred()
+    if not password:
+        sys.exit(f"refused: {SECRET} carries no ABS_ADMIN_PASSWORD")
+    code, body = call("POST", "/login", {"username": user, "password": password})
+    if code != 200:
+        sys.exit(f"login failed {code}: {body}")
+    return body["user"]["token"]      # the token is inside user, not top level
+
+
+def library_id(token, name="Audiobooks"):
+    code, libs = call("GET", "/api/libraries", token=token)
+    if code != 200:
+        sys.exit(f"libraries failed {code}: {libs}")
+    for lib in libs.get("libraries", []):
+        if lib["name"] == name:
+            return lib["id"]
+    sys.exit(f"no library named {name}")
+
+
+def all_item_ids(token, lid):
+    ids, page = [], 0
+    code, one = call("GET", f"/api/libraries/{lid}/items?limit=1", token=token)
+    total = one.get("total", -1) if code == 200 else -1
+    while True:
+        code, body = call("GET", f"/api/libraries/{lid}/items?page={page}&limit=500",
+                          token=token)
+        if code != 200:
+            sys.exit(f"items page {page} failed {code}")
+        results = body.get("results", [])
+        if not results:
+            break
+        ids += [it["id"] for it in results]
+        page += 1
+        if total >= 0 and page * 500 >= total:
+            break
+    return list(dict.fromkeys(ids)), total
+
+
+def cmd_covers(args):
+    copied = missing = exists = 0
+    for line in open(args.list):
+        src, _, dst = line.rstrip("\n").partition("\t")
+        if not dst:
+            continue
+        if os.path.exists(dst):
+            exists += 1
+            continue
+        if not os.path.exists(src):
+            missing += 1
+            continue
+        os.makedirs(os.path.dirname(dst), exist_ok=True)
+        shutil.copy2(src, dst)
+        copied += 1
+    print(f"covers copied: {copied}  already there: {exists}  missing source: {missing}")
+
+
+def cmd_wipe(args, token, lid):
+    ids, total = all_item_ids(token, lid)
+    print(f"library {lid}: {len(ids)} items (total={total})")
+    if not args.apply:
+        print("dry run: nothing deleted (add --apply)")
+        return
+    gone = 0
+    for i, iid in enumerate(ids):
+        code, _ = call("DELETE", f"/api/items/{iid}", token=token)
+        if code == 200:
+            gone += 1
+        elif i < 3:
+            print(f"delete {iid}: {code}")
+        if i % 500 == 499:
+            print(f"  {i + 1}/{len(ids)} deleted")
+    print(f"deleted {gone}/{len(ids)} item rows")
+
+
+def cmd_scan(args, token, lid):
+    code, body = call("POST", f"/api/libraries/{lid}/scan", token=token)
+    print(f"scan: {code} {body if code != 200 else 'accepted'}")
+    if not args.wait:
+        return
+    last = -1
+    while True:
+        time.sleep(30)
+        code, lib = call("GET", f"/api/libraries/{lid}", token=token)
+        code2, one = call("GET", f"/api/libraries/{lid}/items?limit=1", token=token)
+        total = one.get("total", -1) if code2 == 200 else -1
+        fetching = lib.get("isFetchingLibrary") if code == 200 else None
+        print(f"  fetching={fetching} items={total}")
+        if fetching is False and total == last and total >= 0:
+            break
+        last = total
+    print(f"scan settled at {total} items")
+
+
+def cmd_status(token, lid):
+    missing = 0
+    seen, page = set(), 0
+    while True:
+        code, body = call("GET", f"/api/libraries/{lid}/items?page={page}&limit=500",
+                          token=token)
+        results = body.get("results", []) if code == 200 else []
+        if not results:
+            break
+        for it in results:
+            if it["id"] not in seen:
+                seen.add(it["id"])
+                if it.get("isMissing"):
+                    missing += 1
+        page += 1
+    code, lib = call("GET", f"/api/libraries/{lid}", token=token)
+    print(f"items: {len(seen)}  isMissing: {missing}  "
+          f"pending tasks: {lib.get('pendingTasks') if code == 200 else '?'}")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    c = sub.add_parser("covers")
+    c.add_argument("--list", required=True)
+    w = sub.add_parser("wipe")
+    w.add_argument("--apply", action="store_true")
+    s = sub.add_parser("scan")
+    s.add_argument("--wait", action="store_true")
+    sub.add_parser("status")
+    args = ap.parse_args()
+    token = login()
+    lid = library_id(token)
+    if args.cmd == "covers":
+        cmd_covers(args)
+    elif args.cmd == "wipe":
+        cmd_wipe(args, token, lid)
+    elif args.cmd == "scan":
+        cmd_scan(args, token, lid)
+    else:
+        cmd_status(token, lid)
+
+
+if __name__ == "__main__":
+    main()

scripts/audiobookshelf-init.sh

@@ -0,0 +1,311 @@
+#!/usr/bin/env bash
+# audiobookshelf-init.sh — make an Audiobookshelf server usable, from nothing.
+#
+# Why this is a script and not a browser session (live 2026-11-02, Dankserver):
+# a freshly deployed Audiobookshelf answers /healthcheck with 200 while it has no
+# user, no library and no scan — "healthy" says nothing about usefulness, and a
+# service that only works after someone remembers to click through a first-run
+# wizard is not deployed, it is installed. Everything below is the API the React
+# client itself calls, so the same procedure runs unattended and lands in git.
+#
+# The API facts this script depends on, read off the pinned image rather than
+# guessed (dist-server/server/Server.js inside the digest this repo pins):
+#   * the creation route is POST /init with {"newRoot":{"username","password"}}.
+#     There is NO /setup route in 2.37.x — POST /setup answers "Unauthorized",
+#     which reads exactly like an auth fault and is instead a missing route.
+#   * /status (no /api prefix) reports {"isInit": <has a root user>}; every route
+#     under /api needs the bearer token that POST /login returns.
+#   * the library count is NOT a field on the library DTO; the door is
+#     GET /api/libraries/{id}/items -> {"total": N}.
+#
+# The credential is generated once into a mode-600 host file and never enters
+# the repo, the URL, or a process argument list. Same pattern as calibre-server.
+#
+# Single sign-on (authentik) is armed by the same run, and only when the credential
+# file carries ABS_OIDC_CLIENT_SECRET, so a local-only deployment is unchanged by a
+# re-run. These OIDC facts came from the same pinned image, and each one is a trap:
+#   * isOpenIDAuthSettingsValid demands issuer + authorization + token + userinfo +
+#     jwks + clientID + clientSecret + signingAlgorithm at once. Miss one and the
+#     server SILENTLY strips 'openid' from authActiveAuthMethods and falls back to
+#     ['local'] while answering 200 everywhere — so the only honest check is what
+#     /status says AFTER the write, which is what this script does.
+#   * the algorithm is HS256, not the image's RS256 default: no provider on the
+#     fleet's authentik carries a signing key, so the client secret IS the signature
+#     key (scripts/authentik-audiobookshelf-sso.sh). Left at RS256, every id_token
+#     fails verification and the failure looks like a broken IdP.
+#   * the auth block lives on PATCH /api/auth-settings. PATCH /api/settings has a
+#     20-key allowlist (PATCHABLE_SETTINGS_KEYS) with NO auth key in it, so writing
+#     the OIDC config there answers 200 and changes nothing — measured on this host
+#     2026-11-02. The keys /api/auth-settings accepts are exactly its own
+#     authenticationSettings getters; anything else is skipped with a debug line.
+#   * authOpenIDSubfolderForRedirectURLs must be sent explicitly. The 2.17.4 migration only
+#     fills it on a DB that ALREADY had OIDC on, so a database created before SSO leaves it
+#     UNDEFINED and the server builds redirect_uri
+#     "http://host/undefined/auth/openid/callback" — authentik answers the authorize request
+#     with "Redirect URI Error" (measured 2026-11-02). This script derives the value from
+#     ABS_URL's path, so it cannot drift from the unit's ROUTER_BASE_PATH.
+#   * 'local' stays in authActiveAuthMethods on purpose: if authentik is ever down,
+#     the root account must still be able to sign into its own server.
+#
+# usage: audiobookshelf-init.sh          # create what is missing, then scan
+#        ABS_SCAN=0 audiobookshelf-init.sh   # bootstrap without a scan
+#        ABS_OIDC=0 audiobookshelf-init.sh   # never touch the auth methods
+#
+# environment (defaults are the deployed shape):
+#   ABS_URL      server base URL INCLUDING the mounted base path
+#                (http://127.0.0.1:8000/audiobookshelf)
+#   ABS_SECRET   credential file              (/home/dank/.config/secrets/audiobookshelf)
+#   ABS_LIBRARY  library name to ensure        (Audiobooks)
+#   ABS_FOLDER   folder to bind inside the server (/audiobooks)
+#   ABS_SCAN     1 triggers a scan (default), 0 skips it
+#   ABS_OIDC     auto (default) arms SSO when the credential file has a client
+#                secret, 1 requires one, 0 leaves the auth methods alone
+#   ABS_OIDC_ISSUER   authentik issuer URL for this client
+#   ABS_OIDC_CLIENT_ID provider client id (audiobookshelf)
+#   ABS_OIDC_ALG       id_token signing algorithm the IdP actually uses (HS256)
+#   ABS_OIDC_SUBFOLDER override for the redirect-URI subfolder (default: ABS_URL's path)
+#
+# exit 0  every step is satisfied (created or already present)
+# exit 1  a step was refused or the server misbehaved (named on stderr)
+# exit 2  usage / unreadable credential file
+set -euo pipefail
+
+ABS_URL=${ABS_URL:-http://127.0.0.1:8000/audiobookshelf}
+ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf}
+ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks}
+ABS_FOLDER=${ABS_FOLDER:-/audiobooks}
+ABS_SCAN=${ABS_SCAN:-1}
+ABS_OIDC=${ABS_OIDC:-auto}
+ABS_OIDC_ISSUER=${ABS_OIDC_ISSUER:-https://auth.franksdank.cloud/application/o/audiobookshelf/}
+ABS_OIDC_CLIENT_ID=${ABS_OIDC_CLIENT_ID:-audiobookshelf}
+ABS_OIDC_ALG=${ABS_OIDC_ALG:-HS256}
+
+die() { printf 'refused: %s\n' "$*" >&2; exit 1; }
+
+case "$ABS_SCAN" in
+    0 | 1) ;;
+    *) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;;
+esac
+
+case "$ABS_OIDC" in
+    auto | 0 | 1) ;;
+    *) printf 'refused: ABS_OIDC wants auto, 0 or 1, got %s\n' "$ABS_OIDC" >&2; exit 2 ;;
+esac
+
+# ---- credential -------------------------------------------------------------
+umask 077
+if [ ! -f "$ABS_SECRET" ]; then
+    mkdir -p "$(dirname "$ABS_SECRET")"
+    user=${ABS_ADMIN_USER:-frank}
+    # `tr … | head -c 24` is the obvious line and it is a trap under
+    # `set -o pipefail`: head exits at 24 bytes, tr takes SIGPIPE, the pipeline
+    # status is 141 and set -e kills the script before the credential is ever
+    # written (measured 2026-11-02). od reads a FIXED byte count and exits on its
+    # own, so no consumer closes a pipe early; bash slices the length.
+    pass=$(od -An -tx1 -N 32 /dev/urandom | tr -d ' \n')
+    pass=${pass:0:24}
+    [ "${#pass}" -eq 24 ] || die "could not generate a password (no /dev/urandom)"
+    printf 'ABS_ADMIN_USER=%s\nABS_ADMIN_PASSWORD=%s\n' "$user" "$pass" > "$ABS_SECRET"
+    printf 'credential: created %s\n' "$ABS_SECRET"
+fi
+[ -r "$ABS_SECRET" ] || die "$ABS_SECRET exists but is not readable by $(id -un)"
+chmod 600 "$ABS_SECRET"
+
+# ---- everything else, over HTTP ---------------------------------------------
+# python3 rather than curl so the password and the bearer token never appear in
+# any argv (they would be world-readable in /proc/<pid>/cmdline otherwise).
+ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \
+ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" ABS_OIDC="$ABS_OIDC" \
+ABS_OIDC_ISSUER="$ABS_OIDC_ISSUER" ABS_OIDC_CLIENT_ID="$ABS_OIDC_CLIENT_ID" \
+ABS_OIDC_ALG="$ABS_OIDC_ALG" python3 - <<'PY'
+import json, os, sys, urllib.error, urllib.parse, urllib.request
+
+BASE = os.environ["ABS_URL"].rstrip("/")
+LIB = os.environ["ABS_LIBRARY"]
+FOLDER = os.environ["ABS_FOLDER"]
+SCAN = os.environ["ABS_SCAN"] == "1"
+cred = {}
+for line in open(os.environ["ABS_SECRET"]):
+    if "=" in line:
+        k, _, v = line.strip().partition("=")
+        cred[k] = v
+user = cred.get("ABS_ADMIN_USER") or "frank"
+password = cred.get("ABS_ADMIN_PASSWORD") or ""
+if not password:
+    sys.exit("refused: %s has no ABS_ADMIN_PASSWORD" % os.environ["ABS_SECRET"])
+
+
+def call(method, path, body=None, token=None, want_json=True):
+    data = None
+    headers = {"X-Requested-With": "audiobookshelf-init"}
+    if body is not None:
+        data = json.dumps(body).encode()
+        headers["Content-Type"] = "application/json"
+    if token:
+        headers["Authorization"] = "Bearer " + token
+    req = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
+    try:
+        with urllib.request.urlopen(req, timeout=60) as r:
+            raw = r.read()
+            if not raw or not want_json:
+                return r.status, raw
+            try:
+                return r.status, json.loads(raw)
+            except ValueError:
+                # Express answers sendStatus() with the STATUS TEXT as the body:
+                # POST /init and the scan route both reply "OK" (measured on the
+                # pinned 2.37.2 image). A successful call must never be judged by
+                # whether its body happens to parse as JSON.
+                return r.status, raw.decode("utf-8", "replace")[:200]
+    except urllib.error.HTTPError as e:
+        raw = e.read()
+        try:
+            payload = json.loads(raw)
+        except Exception:
+            payload = raw.decode("utf-8", "replace")[:200]
+        return e.code, payload
+    except urllib.error.URLError as e:
+        # Not an HTTP answer at all: nothing listening, DNS dead, connection
+        # refused. HTTPError is a URLError subclass, so this arm must come after.
+        return 0, "unreachable (%s)" % e.reason
+
+
+status, st = call("GET", "/status")
+if status != 200 or not isinstance(st, dict):
+    sys.exit("refused: %s/status answered %s (%s) — is the service up?" % (BASE, status, st))
+if st.get("isInit"):
+    print("account: already registered")
+else:
+    code, body = call("POST", "/init", {"newRoot": {"username": user, "password": password}})
+    if code != 200:
+        sys.exit("refused: POST /init answered %s (%s)" % (code, body))
+    print("account: created (%s)" % user)
+
+code, login = call("POST", "/login", {"username": user, "password": password})
+if code != 200 or not isinstance(login, dict) or not login.get("user", {}).get("token"):
+    sys.exit("refused: login answered %s (%s) — the credential file does not match the server"
+             % (code, login))
+token = login["user"]["token"]
+print("token: obtained (%s)" % login["user"].get("type", "user"))
+
+code, libs = call("GET", "/api/libraries", token=token)
+if code != 200 or not isinstance(libs, dict) or "libraries" not in libs:
+    sys.exit("refused: GET /api/libraries answered %s (%s)" % (code, libs))
+lib = next((l for l in libs["libraries"] if l.get("name") == LIB), None)
+if lib:
+    print("library: exists %s (%s)" % (LIB, lib["id"]))
+else:
+    body = {
+        "name": LIB,
+        "provider": "audiobook",
+        "icon": "book",
+        "folders": [{"fullPath": FOLDER}],
+        "settings": {
+            "coverEncoding": "jpg",
+            "skipMatchingMediaWithAsin": False,
+            "skipMatchingMediaWithIsbn": False,
+            "autoDetectCrossoverAudiobooks": False,
+            "ignoreAudiobookMetadata": False,
+            "useAudiobookFileMetadata": False,
+        },
+        "providerMetadata": {},
+    }
+    code, made = call("POST", "/api/libraries", body, token=token)
+    if code != 200:
+        sys.exit("refused: POST /api/libraries answered %s (%s)" % (code, made))
+    lib = made
+    print("library: created %s (%s) folder %s" % (LIB, lib["id"], FOLDER))
+
+if SCAN:
+    code, body = call("POST", "/api/libraries/%s/scan" % lib["id"], {}, token=token,
+                      want_json=False)
+    if code != 200:
+        sys.exit("refused: scan answered %s (%s)" % (code, body))
+    print("scan: accepted")
+else:
+    print("scan: skipped (ABS_SCAN=0)")
+
+code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=token)
+if code == 200 and isinstance(items, dict):
+    print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)"
+          % items.get("total"))
+
+# ---- single sign-on (authentik) --------------------------------------------------
+# The provider this points at is provisioned by scripts/authentik-audiobookshelf-sso.sh;
+# the two share exactly one value, the client secret in the credential file.
+mode = os.environ["ABS_OIDC"]
+oidc_secret = cred.get("ABS_OIDC_CLIENT_SECRET") or ""
+if mode == "0":
+    print("oidc: skipped (ABS_OIDC=0)")
+elif not oidc_secret:
+    if mode == "1":
+        sys.exit("refused: ABS_OIDC=1 but %s has no ABS_OIDC_CLIENT_SECRET — run "
+                 "scripts/authentik-audiobookshelf-sso.sh first" % os.environ["ABS_SECRET"])
+    print("oidc: not configured (no ABS_OIDC_CLIENT_SECRET in the credential file)"
+          " — local auth only")
+else:
+    client_id = os.environ["ABS_OIDC_CLIENT_ID"]
+    issuer = os.environ["ABS_OIDC_ISSUER"].rstrip("/") + "/"
+    # redirect_uri = <origin><subfolder>/auth/openid/callback, and the subfolder IS the
+    # base path the app is mounted under — taken from ABS_URL so unit and config agree.
+    subfolder = os.environ.get("ABS_OIDC_SUBFOLDER")
+    if subfolder is None:
+        subfolder = urllib.parse.urlparse(os.environ["ABS_URL"]).path.rstrip("/")
+    origin = issuer.split("/application/")[0]
+    body = {
+        "authActiveAuthMethods": ["local", "openid"],
+        "authOpenIDIssuerURL": issuer,
+        "authOpenIDAuthorizationURL": origin + "/application/o/authorize/",
+        "authOpenIDTokenURL": origin + "/application/o/token/",
+        "authOpenIDUserInfoURL": origin + "/application/o/userinfo/",
+        "authOpenIDJwksURL": issuer + "jwks/",
+        "authOpenIDLogoutURL": issuer + "end-session/",
+        "authOpenIDClientID": client_id,
+        "authOpenIDClientSecret": oidc_secret,
+        "authOpenIDTokenSigningAlgorithm": os.environ["ABS_OIDC_ALG"],
+        "authOpenIDSubfolderForRedirectURLs": subfolder,
+        "authOpenIDButtonText": "Sign in with Authentik",
+        # not auto-launch: the login page keeps offering the local form, which is the
+        # break-glass path when the IdP is unreachable.
+        "authOpenIDAutoLaunch": False,
+        # a household user's first SSO sign-in creates their account. The role comes from
+        # the abs_groups claim, which grants the listener role to every signer; only the
+        # authentik admin group escalates. Revoking a listener therefore happens HERE in
+        # ABS (disable or delete the user), never by removing an IdP group membership.
+        "authOpenIDAutoRegister": True,
+        "authOpenIDGroupClaim": "abs_groups",
+        # The app's own scheme lives HERE, never in the IdP: ABS swaps it for
+        # <base>/auth/openid/mobile-redirect before the provider sees a redirect_uri.
+        "authOpenIDMobileRedirectURIs": ["audiobookshelf://oauth"],
+    }
+    code, res = call("PATCH", "/api/auth-settings", body, token=token)
+    if code != 200:
+        sys.exit("refused: PATCH /api/auth-settings answered %s (%s)" % (code, res))
+    # Verify twice: what was stored, and what the server will honour. A config that
+    # misses any of the eight required OIDC values is accepted with 200 and then
+    # silently dropped at the next load, leaving authMethods=['local'].
+    code, stored = call("GET", "/api/auth-settings", token=token)
+    if code != 200 or not isinstance(stored, dict):
+        sys.exit("refused: GET /api/auth-settings answered %s (%s)" % (code, stored))
+    for key, want in (("authOpenIDIssuerURL", issuer),
+                      ("authOpenIDTokenSigningAlgorithm", os.environ["ABS_OIDC_ALG"]),
+                      ("authOpenIDSubfolderForRedirectURLs", subfolder),
+                      ("authOpenIDButtonText", "Sign in with Authentik")):
+        if stored.get(key) != want:
+            sys.exit("refused: stored %s=%r, expected %r" % (key, stored.get(key), want))
+    if sorted(stored.get("authActiveAuthMethods") or []) != ["local", "openid"]:
+        sys.exit("refused: stored authActiveAuthMethods=%r, expected local+openid "
+                 "('openid' is dropped when the block is incomplete)"
+                 % stored.get("authActiveAuthMethods"))
+    code, after = call("GET", "/status")
+    methods = after.get("authMethods") if isinstance(after, dict) else None
+    if not isinstance(methods, list) or "openid" not in methods:
+        sys.exit("refused: /status reports authMethods=%s after a stored config that "
+                 "looks complete — restart the service, the strategy is only armed at "
+                 "load for a config that was invalid when the server read it" % methods)
+    print("oidc: armed (issuer %s | alg %s | claim abs_groups | redirect subfolder %r | "
+          "client %s | secret %s chars | authMethods %s)"
+          % (issuer, os.environ["ABS_OIDC_ALG"], subfolder, client_id, len(oidc_secret),
+             methods))
+PY
+exit 0

scripts/authentik-audiobookshelf-sso.sh

@@ -0,0 +1,294 @@
+#!/usr/bin/env bash
+# authentik-audiobookshelf-sso.sh — provision the authentik side of Audiobookshelf SSO.
+#
+# Why a script: this state lives in authentik's DATABASE, not in any repo file, and a
+# blueprint re-import or a rebuild-from-scratch reverts hand-made provider rows
+# silently. Same reasoning (and same door) as
+# authentik-set-login-session-duration.sh: ssh into Dankclaw, `ak shell` from stdin.
+# Re-run this after such a rebuild.
+#
+# The contract with Audiobookshelf, read off the pinned image (2.37.2,
+# dist-server/server/auth/OidcAuthStrategy.js + objects/settings/ServerSettings.js)
+# rather than a forum, because each of these is a dead-login trap:
+#
+#   * Two URIs per origin ABS is reached at — <base>/auth/openid/callback and
+#     <base>/auth/openid/mobile-redirect — for EVERY entry in BASES (the LAN address
+#     and the public vhost). ABS SWAPS the app's own audiobookshelf://oauth for the
+#     mobile-redirect URL before the IdP ever sees a redirect_uri, so registering the
+#     app scheme with authentik — the obvious move — is what makes phone logins fail.
+#     The app scheme belongs in ABS's authOpenIDMobileRedirectURIs list (its built-in
+#     default), not here. The mobile URI exists on every origin because the app sends
+#     the one belonging to the door it was configured with: the vhost started serving
+#     non-browser clients on 2026-11-03 (it left the guard chain), so a phone pointed
+#     at the public name sends the public mobile-redirect URI, and STRICT matching
+#     answers an unregistered one with "Redirect URI Error".
+#   * <base> CARRIES THE BASE PATH. ABS builds redirect_uri as
+#     <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and that subfolder is
+#     the ROUTER_BASE_PATH the container is mounted at — /audiobookshelf, which is also the only
+#     base path the shipped SPA can boot under. Left unset (the fresh-DB default) the URL
+#     becomes "/undefined/…"; a provider registered without the prefix mismatches the same way
+#     (measured 2026-11-02, "Redirect URI Error"). audiobookshelf-init.sh derives its copy from
+#     ABS_URL, so the two sides each move with one variable.
+#   * ABS requests the scopes "openid profile email <GroupClaim> [<PermsClaim>]": the
+#     CLAIM NAME is a scope name. So authentik needs a scope mapping whose scope_name
+#     is exactly the claim name, or the claim never arrives and the callback dies on
+#     "Group claim <name> not found in userinfo".
+#   * the claim's VALUES must be ABS role names — the callback picks the first of
+#     ['admin','user','guest'] present in the list. Group names like
+#     audiobookshelf_admins are therefore translated by the mapping, never passed
+#     through. A user in neither group gets [] and is denied (fail closed).
+#   * tokens are HS256 here: no provider on this box carries a signing_key (census
+#     2026-11-02), which makes the client secret the signature key. ABS's own default
+#     is RS256, so the ABS side MUST be set to HS256 or every id_token fails
+#     verification.
+#   * provider.authorization_flow must be non-NULL or /application/o/authorize/
+#     answers a 500 with 'NoneType' object has no attribute 'slug' (measured on
+#     grafana, same authentik build).
+#   * provider.grant_types MUST contain authorization_code. authentik 2026.8 defaults an
+#     ORM-created provider to the EMPTY list, and /application/o/authorize/ then refuses the
+#     code flow with oauthlib's generic "The request is otherwise malformed"; the only line
+#     that names the cause is the server log's 'Invalid grant_type for provider'. The
+#     redirect-URI check runs FIRST, so a provider carrying both faults shows only the URI
+#     error until the URI is fixed (measured 2026-11-02, where the login survived the URI fix
+#     and died here).
+#   * property_mappings must carry the built-in OpenID scope mappings ('openid', 'profile',
+#     'email') alongside the claim mapping. A provider holding only the claim mapping grants a
+#     token scoped shorter than the scopes ABS requests, and ABS's /userinfo call then dies
+#     'insufficient_scope (The request requires higher privileges than provided by the access
+#     token)' after authentik has already issued a good code (measured 2026-11-02).
+#
+# Idempotent: every object is created or asserted. Safe to re-run.
+#
+# usage: ABS_OIDC_CLIENT_SECRET=<secret> authentik-audiobookshelf-sso.sh
+#        (the secret is the value stored in
+#         /home/dank/.config/secrets/audiobookshelf on Dankserver — get it with
+#         ssh dankserver awk -F= '/^ABS_OIDC_CLIENT_SECRET=/{print $2}' \
+#             ~/.config/secrets/audiobookshelf)
+#
+# environment:
+#   ABS_OIDC_CLIENT_SECRET  REQUIRED. The provider's client secret.
+#   ABS_OIDC_CLIENT_ID      provider/client id           (audiobookshelf)
+#   ABS_OIDC_GROUPS_CLAIM   claim ABS reads for roles    (abs_groups)
+#   ABS_OIDC_USERNAME       authentik user made admin    (akadmin)
+#   There is deliberately NO user list. The claim grants the LISTENER role to every
+#   authentik account that signs in (2026-11-03), so onboarding a listener is exactly
+#   "create the authentik account". Before this, a household account authenticated at
+#   the IdP and was then refused by ABS for being in no group — a policy decision that
+#   read as a broken login. Revocation lives on the ABS side (disable or delete the
+#   user there): a group can no longer withhold a role the claim always sends.
+#   ABS_E2E_ADMIN           authentik user kept in the ADMIN group for the Playwright gate
+#                           (qa-e2e-admin — tests/e2e/specs/23-audiobookshelf-sso.spec.ts
+#                            signs in as this identity, so a re-run restores the fixture)
+#   ABS_BASE                ABS URL as the browser sees it, base path included
+#                           (http://192.168.0.235:8000/audiobookshelf — the LAN
+#                            origin, reached directly, which is also what the phone
+#                            app is configured with)
+#   ABS_EXTRA_BASES         space-separated additional origins, base path included
+#                           (default: https://audiobooks.franksdank.cloud/audiobookshelf
+#                            — the guarded vhost, dankclaw/Caddyfile).
+#                            ABS builds its redirect_uri from the ORIGIN of the
+#                            request, so each origin the app is served at needs its
+#                            own registered URI — an unregistered origin is a
+#                            redirect_uri_mismatch at the IdP, before any credential.
+#                            Set it empty to register the LAN origin alone.
+#   AUTHENTIK_HOST          ssh target                    (dankclaw)
+set -euo pipefail
+
+SECRET="${ABS_OIDC_CLIENT_SECRET:?set ABS_OIDC_CLIENT_SECRET (see the header)}"
+[ "${#SECRET}" -ge 32 ] || { echo "refused: ABS_OIDC_CLIENT_SECRET is too short to be a secret" >&2; exit 2; }
+
+CLIENT_ID="${ABS_OIDC_CLIENT_ID:-audiobookshelf}"
+CLAIM="${ABS_OIDC_GROUPS_CLAIM:-abs_groups}"
+ADMIN_GROUP="audiobookshelf_admins"
+SSO_USER="${ABS_OIDC_USERNAME:-akadmin}"
+E2E_ADMIN="${ABS_E2E_ADMIN:-qa-e2e-admin}"
+BASE="${ABS_BASE:-http://192.168.0.235:8000/audiobookshelf}"
+EXTRA_BASES="${ABS_EXTRA_BASES:-https://audiobooks.franksdank.cloud/audiobookshelf}"
+HOST="${AUTHENTIK_HOST:-dankclaw}"
+
+# The claim expression: every signer is a listener; only the admin group escalates.
+# Stored verbatim so a later read of the provider shows exactly this policy. The default
+# lives HERE and nowhere else — ABS reads this list off the token, so an account in no
+# Audiobookshelf group still arrives with ["user"] and is auto-registered as a listener
+# instead of being refused after a successful authentication.
+read -r -d '' EXPR <<PYEXPR || true
+groups = {g.name for g in user.groups.all()}
+# Order is deliberate: an admin's claim comes out ["admin","user"], byte-identical to what
+# this mapping emitted before the default existed, because ABS's group-tag matching has never
+# been measured for order-sensitivity and the working admin login is not the place to find out.
+roles = []
+if "$ADMIN_GROUP" in groups:
+    roles.append("admin")
+roles.append("user")
+return {"$CLAIM": roles}
+PYEXPR
+
+# NOTE: the heredoc below is UNQUOTED on purpose — its body interpolates the variables
+# above. The consequence is that a backtick anywhere in that python body is a LOCAL command
+# substitution whose output is spliced into the program before it ever runs, and comments
+# are not exempt, because bash does not know what a python comment is. Write names plainly
+# in the body: measured 2026-11-03, two comments in this body had the shell run
+# audiobookshelf_users and user as commands on every execution of this script.
+ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
+from authentik.core.models import Application, Group, User
+from authentik.flows.models import Flow
+from authentik.providers.oauth2.models import (
+    ClientType, OAuth2Provider, RedirectURI, RedirectURIMatchingMode, RedirectURIType, ScopeMapping)
+
+ADMIN_GROUP = "$ADMIN_GROUP"
+CLAIM, CLIENT_ID, SECRET, BASE = "$CLAIM", "$CLIENT_ID", "$SECRET", "$BASE"
+BASES = [BASE] + """$EXTRA_BASES""".split()
+SSO_USER, E2E_ADMIN = "$SSO_USER", "$E2E_ADMIN"
+EXPR = '''$EXPR'''
+# The expression reads user.groups, NOT user.ak_groups: authentik 2026.8 logs
+# "User.ak_groups is deprecated … use User.groups" on every evaluation of the old
+# name (measured on this run), and a stored mapping that leans on a deprecated
+# accessor is a future upgrade failure sitting inside the login path.
+
+Group.objects.get_or_create(name=ADMIN_GROUP)
+admin_g = Group.objects.get(name=ADMIN_GROUP)
+# The audiobookshelf_users group is retired. Its membership used to be the only door to the
+# listener role; the claim now sends that role to every signer, so the group decides
+# nothing — and a group that decides nothing is a trap for the next reader, who will
+# assume it gates something and start adding people to it.
+for stale in Group.objects.filter(name="audiobookshelf_users"):
+    stale.delete()
+    print("group: retired audiobookshelf_users (the claim grants every signer)")
+
+# --- the claim mapping (scope_name IS the claim name ABS requests) ---------------
+mapping = ScopeMapping.objects.filter(name="audiobookshelf-groups").first()
+if mapping is None:
+    mapping = ScopeMapping.objects.create(name="audiobookshelf-groups", scope_name=CLAIM,
+                                          expression=EXPR,
+                                          description="Audiobookshelf roles: every signer is a listener, the admin group escalates")
+    print("mapping: created (scope_name %s)" % CLAIM)
+elif mapping.scope_name != CLAIM or mapping.expression != EXPR:
+    mapping.scope_name, mapping.expression = CLAIM, EXPR
+    mapping.save(update_fields=["scope_name", "expression"])
+    print("mapping: scope/expression updated (scope_name %s)" % CLAIM)
+else:
+    print("mapping: already current")
+
+# Prove the expression runs and yields an ABS role BEFORE anyone tries to log in:
+# authentik execs the expression inside a function whose context has the name "user".
+_src = "def _probe(user, request=None):\n" + "".join("    %s\n" % ln for ln in EXPR.splitlines())
+_ns = {}
+exec(_src, _ns)
+
+# --- provider --------------------------------------------------------------------
+# One callback pair per origin (LAN address and the public vhost): ABS derives the
+# browser callback from the request origin and the mobile callback from the door the
+# app was configured with, so each origin needs both or that door cannot sign in.
+uris = [RedirectURI(matching_mode=RedirectURIMatchingMode.STRICT, url="%s%s" % (b, p))
+        for b in BASES
+        for p in ("/auth/openid/callback", "/auth/openid/mobile-redirect")]
+authz_flow = Flow.objects.get(slug="default-provider-authorization-implicit-consent")
+
+# Three things beyond the name, each one a silent dead login: the grant-type list (2026.8
+# defaults it empty), the built-in OpenID scope mappings beside the claim mapping, and the
+# redirect URIs with the base path baked in.
+GRANTS = ["authorization_code", "refresh_token"]
+BUILTIN_SCOPES = ["authentik default OAuth Mapping: OpenID 'openid'",
+                  "authentik default OAuth Mapping: OpenID 'profile'",
+                  "authentik default OAuth Mapping: OpenID 'email'"]
+scoped = {m.pk: m for m in ScopeMapping.objects.filter(name__in=BUILTIN_SCOPES)}
+scoped[mapping.pk] = mapping
+_have = {m.name for m in scoped.values()}
+if len(_have) < len(BUILTIN_SCOPES):
+    print("scope mappings: NOT PROVIDED by this authentik install: %s"
+          % ", ".join(sorted(set(BUILTIN_SCOPES) - _have)))
+
+p = OAuth2Provider.objects.filter(name="audiobookshelf").first()
+if p is None:
+    p = OAuth2Provider(name="audiobookshelf", client_id=CLIENT_ID,
+                       client_type=ClientType.CONFIDENTIAL, client_secret=SECRET,
+                       authorization_flow=authz_flow, grant_types=list(GRANTS))
+    p.redirect_uris = uris
+    p.save()
+    p.property_mappings.set(list(scoped.values()))
+    print("provider: created (grant_types %s)" % GRANTS)
+else:
+    changed = []
+    if p.client_id != CLIENT_ID:
+        p.client_id = CLIENT_ID
+        changed.append("client_id")
+    if p.client_secret != SECRET:
+        p.client_secret = SECRET
+        changed.append("client_secret")
+    if p.authorization_flow_id != authz_flow.pk:
+        p.authorization_flow = authz_flow
+        changed.append("authorization_flow")
+    if list(p.grant_types) != GRANTS:
+        p.grant_types = list(GRANTS)
+        changed.append("grant_types")
+    if [(u.url, str(u.matching_mode)) for u in p.redirect_uris] != \
+       [(u.url, str(u.matching_mode)) for u in uris]:
+        p.redirect_uris = uris
+        changed.append("redirect_uris")
+    if changed:
+        p.save()
+    if sorted(m.pk for m in p.property_mappings.all()) != sorted(scoped):
+        p.property_mappings.set(list(scoped.values()))
+        changed.append("property_mappings")
+    print("provider: asserted (%s)" % (", ".join(changed) or "nothing to change"))
+
+# --- application (the launchpad entry; binds the provider) ------------------------
+app = Application.objects.filter(slug="audiobookshelf").first()
+if app is None:
+    app = Application.objects.create(name="audiobookshelf", slug="audiobookshelf",
+                                     provider=p, meta_launch_url=BASE)
+    print("application: created")
+elif app.provider_id != p.pk or app.meta_launch_url != BASE:
+    app.provider, app.meta_launch_url = p, BASE
+    app.save(update_fields=["provider", "meta_launch_url"])
+    print("application: asserted")
+else:
+    print("application: already bound")
+
+# --- membership --------------------------------------------------------------------
+root = User.objects.get(username=SSO_USER)
+root.groups.add(admin_g)
+# The deploy gate's Playwright spec signs in as this identity, so the admin fixture is
+# provisioned with the rest: a blueprint re-import that drops it fails the gate, not a phone.
+e2e = User.objects.filter(username=E2E_ADMIN).first()
+if e2e is None:
+    print("e2e fixture: authentik user %s absent — the e2e SSO spec will fail" % E2E_ADMIN)
+else:
+    e2e.groups.add(admin_g)
+    print("e2e fixture: %s in %s" % (E2E_ADMIN, ADMIN_GROUP))
+print("admin: %s in %s" % (SSO_USER, ADMIN_GROUP))
+# The default is printed at provision time, not merely asserted: an expression that regressed
+# to returning nothing for a non-member refuses every household login, and this is where a
+# re-run notices it. The population matters — authentik ships an AnonymousUser row, a
+# superuser, and ak-outpost-* service identities, and proving the default on one of those says
+# nothing about the friend who logs in. Prefer the fleet's plain-user marker group (owned by
+# serverAppConfigs-docs/Infrastructure/Authentik.md; a preference only, since that group
+# belongs to other consumers), which normally lands on qa-e2e — the identity the e2e gate
+# signs in as. is_superuser is a python property on authentik's User, not a queryable field.
+pool = User.objects.filter(is_active=True).exclude(groups__name__in=[ADMIN_GROUP])
+pool = pool.exclude(username__in=[root.username, E2E_ADMIN, "AnonymousUser"])
+pool = [u for u in pool.order_by("username")
+        if not u.is_superuser and not u.username.startswith("ak-outpost-")]
+household = [u for u in pool if any(g.name == "Non-Admin Users" for g in u.groups.all())]
+for u in (household or pool)[:3]:
+    print("default probe: %s %s -> %s" % (
+        u.username, sorted(g.name for g in u.groups.all()),
+        _ns["_probe"](user=u, request=None)))
+
+# --- verification: the claim the callback will actually receive ---------------------
+for probe in [root] + ([e2e] if e2e else []):
+    print("claim for %s -> %s" % (probe.username, _ns["_probe"](user=probe, request=None)))
+p.refresh_from_db()
+print("provider %s | client_id %s | flow %s | grants %s | uris %s" % (
+    p.name, p.client_id, p.authorization_flow.slug, list(p.grant_types),
+    [u.url for u in p.redirect_uris]))
+print("scopes: %s" % sorted(m.scopemapping.scope_name for m in p.property_mappings.all()))
+PYEOF
+
+cat <<DONE
+
+Next: arm the Audiobookshelf side (issuer, client, HS256, claim) —
+  ssh dankserver 'ABS_OIDC=1 bash /home/dank/serverAppConfigs/scripts/audiobookshelf-init.sh'
+It reads the same secret from /home/dank/.config/secrets/audiobookshelf and verifies
+/status answers openid among its auth methods.
+DONE

scripts/authentik-proxy-provider-defaults.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+# Census and repair the OAuth2 defaults on every authentik PROXY provider.
+#
+# A proxy provider built by hand in `ak shell` never runs
+# ProxyProvider.set_oauth_defaults() — the REST serializer calls it, the ORM does
+# not — so the row looks perfectly configured (external_host, internal_host,
+# application bound, embedded-outpost claim present) while three OAuth facts are
+# blank. Each one kills the guarded login in a different, unrelated-looking way,
+# all BEFORE any credential is asked for:
+#
+#   * redirect_uris empty -> authentik answers its own page:
+#     "Redirect URI Error — The request fails due to a missing, invalid, or
+#     mismatching redirection URI (redirect_uri)". Measured on the Audiobookshelf
+#     guard provider, 2026-11-03.
+#   * grant_types empty -> the authorize endpoint refuses the outpost's
+#     authorization-code request with a bodiless HTTP 400 (Chromium renders
+#     "This page isn't working / HTTP ERROR 400"). Same provider, one step
+#     later, after the URIs were filled in by hand.
+#   * authorization_flow NULL -> the authorize view crashes building its flow
+#     planner: AttributeError 'NoneType' object has no attribute 'slug', HTTP
+#     500. That is what authentik-grafana-sso-fix.sh exists for.
+#
+# So: a guard that shows ANY of those three pages is not an Anubis problem and
+# not a Caddy problem — run this script before touching the route.
+#
+# Repair calls the VENDOR's own set_oauth_defaults() (grant_types = authz code +
+# client credentials + password, confidential client, the stock scope mappings,
+# redirect_uris derived from external_host by _get_callback_url) and points the
+# authorization flow at default-provider-authorization-implicit-consent, the flow
+# every working provider here uses. Providers that already match are untouched,
+# so a deliberate per-provider deviation is never silently flattened.
+#
+# This is authentik DB state. It is NOT in any deployable file: a blueprint
+# re-import, or an ak-shell re-creation of a provider, reverts these facts and
+# the symptom returns verbatim. Re-run after any such rebuild. Idempotent.
+#
+# Usage: authentik-proxy-provider-defaults.sh [--check]
+#          no argument  repair what drifts, print one line per provider
+#          --check       census only, exit 1 if anything drifts (gate use)
+#        override host: AUTHENTIK_HOST=…
+set -euo pipefail
+HOST="${AUTHENTIK_HOST:-dankclaw}"
+MODE="${1:-}"
+case "$MODE" in
+    "" | --check) : ;;
+    *) echo "usage: $(basename "$0") [--check]" >&2; exit 2 ;;
+esac
+
+ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
+from authentik.flows.models import Flow
+from authentik.providers.proxy.models import ProxyProvider, _get_callback_url
+
+CHECK = $( [ "$MODE" = "--check" ] && echo True || echo False )
+FLOW = "default-provider-authorization-implicit-consent"
+WANT_GRANTS = ["authorization_code", "client_credentials", "password"]
+
+flow = Flow.objects.filter(slug=FLOW).first()
+if not flow and not CHECK:
+    raise SystemExit("flow %s is missing; nothing was changed" % FLOW)
+
+drift = 0
+for p in ProxyProvider.objects.all().order_by("pk"):
+    want_uris = [u.url for u in _get_callback_url(p.external_host)]
+    have_uris = [u.url for u in p.redirect_uris]
+    bad = []
+    if [str(g) for g in p.grant_types] != WANT_GRANTS:
+        bad.append("grant_types=%s" % [str(g) for g in p.grant_types])
+    if have_uris != want_uris:
+        bad.append("redirect_uris=%s" % have_uris)
+    if p.authorization_flow_id is None:
+        bad.append("authorization_flow=None")
+    if not bad:
+        print("ok   %s (ext %s)" % (p.name, p.external_host))
+        continue
+    drift += 1
+    if CHECK:
+        print("DRIFT %s: %s" % (p.name, "; ".join(bad)))
+        continue
+    print("fix  %s: %s" % (p.name, "; ".join(bad)))
+    p.set_oauth_defaults()
+    if p.authorization_flow_id is None:
+        p.authorization_flow = flow
+    p.save()
+    after = ProxyProvider.objects.get(pk=p.pk)
+    print("     -> grants %s, %d redirect URIs, flow %s"
+          % ([str(g) for g in after.grant_types], len(after.redirect_uris),
+             after.authorization_flow and after.authorization_flow.slug))
+
+print("providers checked, drift: %d" % drift)
+raise SystemExit(1 if (CHECK and drift) else 0)
+PYEOF
+
+echo "Verify: walk the guarded vhost in a browser — Anubis solves, authentik"
+echo "renders its login form (no Redirect URI Error, no HTTP 400), and the"
+echo "callback lands back on the app."

scripts/deploy-plan.py

@@ -60,6 +60,7 @@ SERVICE_ORDER = (
     "calibre-server",
     "plex",
     "tautulli",
+    "audiobookshelf",
     "prowlarr",
     "qbittorrent",
     "radarr",
@@ -123,6 +124,7 @@ DANKSERVER_SERVICES = {
     "calibre-server",
     "plex",
     "tautulli",
+    "audiobookshelf",
     "prowlarr",
     "qbittorrent",
     "radarr",
@@ -155,13 +157,14 @@ DANKCLAW_COMPOSE_SERVICES = set()
 # Quadlet unit now — the last compose services were the hermes trio, whose units
 # have owned their containers since 2026-09-12 and whose compose blocks were
 # dead weight that could only create a duplicate container.
+# jellyfin left the Dankserver file on 2026-10-30 for the same reason: its Quadlet
+# has owned the container since the cutover, so a compose edit must not restart it.
 
 DANKSERVER_COMPOSE_SERVICES = {
     "mariadb",
     "redis",
     "romm",
     "copyparty",
-    "jellyfin",
     "calibre-web",
 }
 COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
@@ -419,8 +422,14 @@ def services_for_path(path: str, target: str) -> set[str]:
                 "copyparty": {"copyparty"},
                 "plex": {"plex"},
                 "tautulli": {"tautulli"},
+                "audiobookshelf": {"audiobookshelf"},
             }
             _add(services, *quadlet_services.get(parts[1], set()))
+        if len(parts) >= 2 and parts[1] == "host":
+            # dankserver/host/systemd/media-meta-view.mount is the overlay view the
+            # media servers bind (scripts/media-view.sh installs it on the host),
+            # so editing it restarts jellyfin and plex and nothing else.
+            _add(services, "jellyfin", "plex")
         if len(parts) >= 2 and parts[1] == "observability":
             # Dankserver's node-exporter became a Quadlet unit on 2026-09-16
             # (serverAppConfigs-docs/Infrastructure/Observability.md): its
@@ -466,6 +475,15 @@ def services_for_path(path: str, target: str) -> set[str]:
             # that service (target_services then keeps it only on the host being
             # planned). Editing the collector restarts the running exporter.
             _add(services, "gpu-exporter")
+        if name == "media-view.sh":
+            # The installer is what creates, mounts and reaps the view both media
+            # servers read, so a change to it is a restart of that pair.
+            _add(services, "jellyfin", "plex")
+        if name == "jellyfin-providers.sh":
+            # The deployer runs this before jellyfin restarts, so a change to
+            # the installer is a change to the provider state Jellyfin loads at
+            # start-up: it selects jellyfin and nothing else.
+            _add(services, "jellyfin")
         if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
             # llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
             # files here; each one selects frank-zealot only. The ik_llama.cpp

scripts/deploy-remote.sh

@@ -95,6 +95,7 @@ service_is_allowed() {
         dankclaw:frank-tts-engine|dankclaw:frank-tts|\
         dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
         dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
+        dankserver:audiobookshelf|\
         dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
         dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
         dankserver:frank-archon|dankserver:node-exporter|dankserver:cadvisor|dankserver:gpu-exporter|\
@@ -322,11 +323,13 @@ copy_alloy_config() {
     # The repo config carries the token GRAFANA_HOST_LABEL; the installed copy
     # carries the host's name, so Loki streams say WHICH machine's journal a
     # line came from without a second config file per host.
-    # *_CONF_ROOT are test seams: the bats cases execute these copiers for
-    # real against a sandbox with a stubbed run_privileged.
+    # *_CONF_ROOT and ALLOY_CONFIG_SRC are test seams: the bats cases execute
+    # these copiers for real against a sandbox with a stubbed run_privileged;
+    # ALLOY_CONFIG_SRC lets the refusal case feed a deliberately broken config.
     local root="${ALLOY_CONF_ROOT:-/opt/alloy}" tmp
+    local src="${ALLOY_CONFIG_SRC:-shared/alloy/config.alloy}"
     tmp="$(mktemp)"
-    sed "s/GRAFANA_HOST_LABEL/$TARGET/g" shared/alloy/config.alloy > "$tmp"
+    sed "s/GRAFANA_HOST_LABEL/$TARGET/g" "$src" > "$tmp"
     if ! grep -q "host = \"$TARGET\"," "$tmp" || grep -q "GRAFANA_HOST_LABEL" "$tmp"; then
         echo "alloy config: host-label substitution failed" >&2
         rm -f "$tmp"
@@ -1202,7 +1205,80 @@ deploy_romm_stack() {
     systemctl --user restart "$service.service"
 }
 
+ensure_media_view() {
+    # Both media servers read their library through the SSD-backed overlay view
+    # (scripts/media-view.sh, dankserver/host/systemd/media-meta-view.mount): the
+    # array below stays read-only and every sidecar Jellyfin writes lands on the
+    # NVMe.
+    # Raise it BEFORE either container starts — a start against a bare view
+    # directory would let a scan clear the library database. Each unit gates on
+    # scripts/wait-for-mount.sh, which waits and then refuses the start. A
+    # relayout must be followed by `media-view.sh reap`
+    # (see the Media Relayout page), which this is not.
+    # A fresh host has no upper base, and `install` refuses to invent one — so
+    # without this the media deploy dies on a rebuilt machine with
+    # "put it on the SSD first" (measured 2026-10-31).
+    run_privileged bash scripts/media-view.sh ensure-base
+    run_privileged bash scripts/media-view.sh install
+    bash scripts/media-view.sh verify
+}
+
+ensure_user_io_delegate() {
+    # The media servers ask for an I/O share (IOWeight= in their quadlets) so a
+    # library scan cannot starve playback or the array's other reader. Two system
+    # drop-ins are what make that request real, and both are host state, so the
+    # deployer places them:
+    #   user.slice.d/io-accounting.conf        io enters user.slice's subtree
+    #   user@1000.service.d/io-delegate.conf   the user manager gets the controller
+    # Without the first, `io` never appears in user.slice/cgroup.subtree_control,
+    # so no container under it can have an io.weight file at all; without the
+    # second the user manager never asks for the controller. systemd hands
+    # user@.service cpu/memory/pids by default and drops a line it cannot honour
+    # without a word — IOWeight= on its own is inert config.
+    local src dst
+    for src in \
+        dankserver/host/systemd/user.slice.d/io-accounting.conf \
+        dankserver/host/systemd/user@1000.service.d/io-delegate.conf; do
+        dst=/etc/systemd/system/${src#dankserver/host/systemd/}
+        run_privileged mkdir -p "$(dirname "$dst")"
+        if ! run_privileged cmp -s "$src" "$dst"; then
+            run_privileged install -m 644 "$src" "$dst"
+            run_privileged systemctl daemon-reload
+        fi
+    done
+    # Accounting lands immediately; the delegation is read when user@1000.service
+    # starts, and restarting that unit would kill every rootless container on this
+    # host. So the share becomes real at the next reboot — say so out loud rather
+    # than let a quadlet line read as live config.
+    if ! systemctl show user@1000.service -p DelegateControllers | grep -qw io; then
+        echo "NOTE: io is not delegated to user@1000.service yet - IOWeight= is"
+        echo "NOTE: inert until the next reboot (see [[Quadlet Lifecycle]])"
+    fi
+}
+
+ensure_provider_keys() {
+    # Jellyfin's metadata-provider keys live in the container-owned plugin
+    # configuration under the data root, never in git; scripts/jellyfin-providers.sh
+    # is what places them, reading the per-service host env file (the fleet
+    # convention: grafana, caddy, pihole, copyparty, romm, seerr) that holds
+    # TMDB_API_KEY= and TVDB_API_KEY=, mode 600.
+    # Missing keys mean a keyless provider — a degraded library, not a broken
+    # deploy — so this warns rather than failing the restart. But when the keys
+    # ARE placed it must happen before the restart: the plugin and its keys are
+    # read at start-up, so a restart first would boot the old state.
+    local keys=/home/dank/Containers/jellyfin/provider-keys
+    if [ -f "$keys" ]; then
+        bash scripts/jellyfin-providers.sh install
+    else
+        echo "WARN: $keys is absent - provider keys NOT placed"
+        echo "WARN: create it (TMDB_API_KEY=, TVDB_API_KEY=, chmod 600) and re-deploy jellyfin"
+    fi
+}
+
 deploy_jellyfin() {
+    ensure_media_view
+    ensure_user_io_delegate
+    ensure_provider_keys
     cp "dankserver/jellyfin/quadlet/jellyfin.container" "$quadlet_unit_dir/"
     systemctl --user daemon-reload
     if ! systemctl --user is-active -q jellyfin.service; then
@@ -1227,6 +1303,37 @@ deploy_calibre_web() {
     systemctl --user restart calibre-web.service
 }
 
+# Audiobookshelf keeps its whole state (SQLite library, accounts, progress,
+# cached covers) on the NVMe under /home/dank/Containers/audiobookshelf; the
+# audio is a read-only bind off the array, so the only thing the deployer has to
+# create is the two state roots. mkdir names the owner — letting podman invent
+# the directories leaves the SQLite file unwritable by the container user on a
+# rootless first boot. No `systemctl --user enable`: a generated Quadlet unit
+# cannot be enabled, and WantedBy=default.target is what carries it across a
+# reboot.
+deploy_audiobookshelf() {
+    cp "dankserver/audiobookshelf/quadlet/audiobookshelf.container" "$quadlet_unit_dir/"
+    systemctl --user daemon-reload
+    mkdir -p /home/dank/Containers/audiobookshelf/config \
+             /home/dank/Containers/audiobookshelf/metadata
+    systemctl --user restart audiobookshelf.service
+    # /healthcheck answers once the HTTP server is bound, at the root whatever
+    # ROUTER_BASE_PATH says (measured on both base paths); the library scan that
+    # follows runs in the background and must not gate the deploy.
+    wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90
+    # A server with no root user and no library is "healthy" and useless, so the
+    # bootstrapper runs after the health gate and its failure fails the deploy:
+    # it creates the account (credential in the mode-600 host secret file), the
+    # Audiobooks library over the /audiobooks bind, and fires the scan. Every
+    # step is idempotent, so a re-deploy costs one incremental scan.
+    # The bootstrapper has to speak the same base path the unit mounts (the API, the
+    # login route and the OIDC redirect subfolder all hang off it), so the URL is read
+    # out of the unit file here instead of being restated in the script's default.
+    abs_base=$(awk -F= '/^Environment=ROUTER_BASE_PATH=/{print $3}' \
+        dankserver/audiobookshelf/quadlet/audiobookshelf.container)
+    ABS_URL="http://127.0.0.1:8000${abs_base}" scripts/audiobookshelf-init.sh
+}
+
 # The content server's single admin account and the desktop's basic-auth
 # credential (same pair) are bootstrapped exactly once: the env file is the
 # plaintext store (mode 600), the userdb's pw column is plaintext too, so
@@ -1290,6 +1397,8 @@ deploy_copyparty() {
 }
 
 deploy_plex() {
+    ensure_media_view
+    ensure_user_io_delegate
     cp "dankserver/plex/quadlet/plex.container" "$quadlet_unit_dir/"
     systemctl --user daemon-reload
     if ! systemctl --user is-active -q plex.service; then
@@ -2101,6 +2210,7 @@ deploy_service() {
         copyparty) deploy_copyparty ;;
         plex) deploy_plex ;;
         tautulli) deploy_tautulli ;;
+        audiobookshelf) deploy_audiobookshelf ;;
         alloy) deploy_alloy ;;
         prometheus|grafana|loki|node-exporter|cadvisor|alertmanager|blackbox-exporter) deploy_observability "$service" ;;
         gpu-exporter) deploy_gpu_exporter ;;

scripts/deploy.sh

@@ -148,6 +148,7 @@ valid_service() {
         dankclaw:frank-dragoon|dankclaw:frank-zealot|\
         dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
         dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
+        dankserver:audiobookshelf|\
         dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
         dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|dankserver:cadvisor|dankserver:gpu-exporter|\
         dankserver:prowlarr|dankserver:frank-archon|\

scripts/jellyfin-providers.sh

@@ -0,0 +1,462 @@
+#!/usr/bin/env bash
+# Metadata-provider keys for Jellyfin on Dankserver, placed from the service's
+# host env file with the plugin artifact pinned by content hash.
+#
+# Jellyfin 12 ships TMDb, OMDb, StudioImages and MusicBrainz as CORE plugins:
+# they have no DLL, only a configuration file under the data root, and that
+# file is created by the running container. So the fleet's provider state lives
+# in two places the repo does not track —
+#
+#   plugins/configurations/Jellyfin.Plugin.Tmdb.xml   <TmdbApiKey /> (empty = keyless)
+#   plugins/configurations/Jellyfin.Plugin.Tvdb.xml   <ApiKey />     (needs the plugin)
+#
+# and both are owned by the container user. This tool is the only thing that
+# writes them: it is declarative, idempotent, and it never prints a key, so it
+# is safe to run while someone watches.
+#
+# Keys never enter git and never enter argv (a value in argv is world-readable
+# through /proc/<pid>/cmdline). They live in the per-service host env file that
+# is the fleet's convention for container credentials (grafana, caddy, pihole,
+# copyparty, romm, seerr), mode 600:
+#
+#   /home/dank/Containers/jellyfin/provider-keys
+#       TMDB_API_KEY=<key>
+#       TVDB_API_KEY=<key>
+#       TVDB_PIN=<pin>        # optional: only a subscriber-shaped TVDB key
+#                             # needs one; a licensed key does not
+#
+# TVDB itself is not core in 12 (the metadata provider was cut in 10.9), so it
+# is installed from the official catalog, pinned three ways: sha256 of the
+# artifact, the plugin's own guid, and its version. A matching sha proves only
+# that the bytes are the bytes fetched — the guid check is what says it is the
+# official plugin, the version check is what says it is the one this file was
+# reviewed against.
+#
+# Subcommands
+#   plan      print the resolved shape and every pending change, touch nothing
+#   install   fetch+verify the plugin, then write the keys (backing up first)
+#   verify    prove the plugin and both keys are in place, and that no library
+#             is a silent file index (providers off / image fetchers empty)
+#
+# Nothing here restarts Jellyfin: the deployer owns that
+# (`scripts/deploy.sh dankserver --service jellyfin`), and the plugin is read at
+# start-up, so a key written while the server runs is not live until then.
+set -euo pipefail
+
+# Jellyfin's data root on Dankserver (the rootless quadlet's config volume).
+JF_ROOT="${JF_ROOT:-/home/dank/Containers/jellyfin/config}"
+PROV_KEYS="${PROV_KEYS:-/home/dank/Containers/jellyfin/provider-keys}"
+TRASH_DIR="${TRASH_DIR:-$HOME/.AGENTTRASH/$(date +%Y%m%d)-jellyfin-providers}"
+
+PLUGINS="$JF_ROOT/plugins"
+CONF="$PLUGINS/configurations"
+TMDB_XML="$CONF/Jellyfin.Plugin.Tmdb.xml"
+TVDB_XML="$CONF/Jellyfin.Plugin.Tvdb.xml"
+
+# repo.jellyfin.org catalog entry for TheTVDB (owner: jellyfin), version 24.0.0.0,
+# targetAbi 12.0.0.0 — published 2026-09-09, verified against Jellyfin 12.4.0.
+TVDB_PLUGIN_URL="${TVDB_PLUGIN_URL:-https://repo.jellyfin.org/files/plugin/thetvdb/thetvdb_24.0.0.0.zip}"
+TVDB_PLUGIN_SHA256="${TVDB_PLUGIN_SHA256:-6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319}"
+TVDB_PLUGIN_GUID="${TVDB_PLUGIN_GUID:-a677c0da-fac5-4cde-941a-7134223f14c8}"
+TVDB_PLUGIN_VERSION="${TVDB_PLUGIN_VERSION:-24.0.0.0}"
+PLUGIN_DIR="thetvdb_${TVDB_PLUGIN_VERSION}"
+TVDB_DLL="Jellyfin.Plugin.Tvdb.dll"
+
+die() {
+    printf 'jellyfin-providers: %s\n' "$*" >&2
+    exit 1
+}
+
+usage() {
+    printf 'usage: %s plan|install|verify\n' "$0" >&2
+    exit 2
+}
+
+# check_keys — refuse anything that is not a readable private file. An open
+# mode is an error, not a warning: a provider key readable by every user on the
+# box is the failure this check exists to catch.
+check_keys() {
+    [ -f "$PROV_KEYS" ] ||
+        die "$PROV_KEYS not found (create it with TMDB_API_KEY= and TVDB_API_KEY=, then chmod 600)"
+    local mode
+    mode="$(stat -c '%a' "$PROV_KEYS")"
+    case "$mode" in
+    600 | 400) ;;
+    *) die "$PROV_KEYS is mode $mode; provider keys must be 600" ;;
+    esac
+}
+
+# keys_value <VAR> — the value of one variable in the env file, or nothing.
+keys_value() {
+    awk -v k="$1" -F= '$1 == k { sub(/^[^=]*=/, ""); print; exit }' "$PROV_KEYS"
+}
+
+# keys_len <VAR> — the length, and a refusal naming the variable when absent.
+# The value itself is never printed.
+keys_len() {
+    local v
+    v="$(keys_value "$1")"
+    [ -n "$v" ] || die "$PROV_KEYS has no $1= line"
+    printf '%s' "${#v}"
+}
+
+# xml_value <file> <element> — the element's text, empty when it is absent or
+# self-closing.
+xml_value() {
+    python3 - "$1" "$2" <<'PYEOF'
+import re, sys
+src = open(sys.argv[1], encoding="utf-8").read()
+m = re.search(r"<%s(?:\s*/>|>(.*?)</%s>)" % (sys.argv[2], sys.argv[2]), src, re.S)
+print(m.group(1).strip() if (m and m.group(1)) else "")
+PYEOF
+}
+
+# xml_secret <file> <element> <var> — set <element> to that variable from the
+# env file. The python process reads the env file itself, so the value never
+# appears in argv.
+xml_secret() {
+    python3 - "$1" "$2" "$3" "$PROV_KEYS" <<'PYEOF'
+import os, re, sys, tempfile
+from xml.sax.saxutils import escape
+
+xml_path, elem, var, keys_path = sys.argv[1:5]
+val = ""
+for line in open(keys_path, encoding="utf-8"):
+    key, sep, rest = line.partition("=")
+    if sep and key.strip() == var:
+        val = rest.strip()
+        break
+if not val:
+    raise SystemExit(f"{keys_path} has no {var}= line")
+
+src = open(xml_path, encoding="utf-8").read()
+pat = re.compile(r"([ \t]*)<%s(?:\s*/>|>.*?</%s>)" % (elem, elem), re.S)
+text = escape(val)
+if pat.search(src):
+    out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
+else:
+    # The plugin owns this file and writes it under ITS OWN root element
+    # (TheTVDB's is <PluginConfiguration>), and its serializer omits a
+    # null-valued setting entirely — so the element being filled can be
+    # absent outright, and a hardcoded </Configuration> anchor then matches
+    # nothing. Anchor on the document's real root instead.
+    out = src
+    root = re.search(r"<([A-Za-z_][\w.:-]*)[^>]*>", src)
+    if root and ("</%s>" % root.group(1)) in src:
+        name = root.group(1)
+        out = src.replace(
+            "</%s>" % name, f"  <{elem}>{text}</{elem}>\n</{name}>", 1
+        )
+if out == src:
+    raise SystemExit(f"{xml_path}: no place to write <{elem}>")
+
+d, base = os.path.split(xml_path)
+fd, tmp = tempfile.mkstemp(prefix=base + ".", dir=d)
+with os.fdopen(fd, "w", encoding="utf-8") as fh:
+    fh.write(out)
+os.chmod(tmp, os.stat(xml_path).st_mode & 0o777)
+os.replace(tmp, xml_path)
+PYEOF
+}
+
+# plugin_meta <zip> — "guid version" out of the artifact's own meta.json.
+plugin_meta() {
+    python3 - "$1" <<'PYEOF'
+import json, sys, zipfile
+meta = json.loads(zipfile.ZipFile(sys.argv[1]).read("meta.json").decode("utf-8"))
+print(meta.get("guid", ""), meta.get("version", ""))
+PYEOF
+}
+
+check_root() {
+    # Never fabricate a data root: creating plugins/ beside a typo'd path would
+    # install a plugin Jellyfin never loads and hide the operator's mistake.
+    [ -d "$CONF" ] || die "no Jellyfin configuration directory at $CONF (is JF_ROOT right?)"
+}
+
+check_keys_report() {
+    check_keys
+    # A refusal inside a command substitution used as a printf argument would
+    # be swallowed, so the guard runs as its own statement.
+    keys_len TMDB_API_KEY >/dev/null
+    keys_len TVDB_API_KEY >/dev/null
+    printf '= keys read from %s (mode %s)\n' "$PROV_KEYS" "$(stat -c '%a' "$PROV_KEYS")"
+}
+
+# The plugin's state, as one line in the shared vocabulary.
+plugin_state() {
+    if [ -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        printf '= %s already installed\n' "$PLUGIN_DIR"
+        return 0
+    fi
+    local other
+    other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
+    if [ -n "$other" ]; then
+        printf '+ install %s (a different version is present: %s)\n' "$PLUGIN_DIR" "$other"
+        return 0
+    fi
+    printf '+ install %s from %s\n' "$PLUGIN_DIR" "${TVDB_PLUGIN_URL#*://}"
+}
+
+tmdb_state() {
+    if [ ! -f "$TMDB_XML" ]; then
+        printf '! %s is missing — save the TMDb plugin settings once in the UI, then re-run\n' "$TMDB_XML" >&2
+        return 1
+    fi
+    local now want
+    now="$(xml_value "$TMDB_XML" TmdbApiKey)"
+    want="$(keys_value TMDB_API_KEY)"
+    if [ -n "$now" ] && [ "$now" = "$want" ]; then
+        printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
+    elif [ -n "$now" ]; then
+        printf '+ replace the TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
+    else
+        printf '+ set TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
+    fi
+}
+
+tvdb_state() {
+    local now
+    if [ -f "$TVDB_XML" ]; then
+        now="$(xml_value "$TVDB_XML" ApiKey)"
+        if [ -n "$now" ]; then
+            printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
+        else
+            printf '+ fill the empty ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
+        fi
+    else
+        printf '+ create Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
+    fi
+}
+
+write_plugin() {
+    local tmp guid version other
+    other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
+    [ -z "$other" ] ||
+        die "another TheTVDB version is already installed ($other); move it to the trash first — two copies of one plugin means two loads"
+
+    tmp="$(mktemp "${TMPDIR:-/tmp}/thetvdb.XXXXXX.zip")"
+    # The pins are checked before anything is unpacked, so a bad artifact never
+    # reaches the plugins directory.
+    curl -fsSL --max-time 300 -o "$tmp" "$TVDB_PLUGIN_URL" ||
+        { rm -f "$tmp"; die "could not fetch $TVDB_PLUGIN_URL"; }
+    local got
+    got="$(sha256sum "$tmp" | cut -d' ' -f1)"
+    if [ "$got" != "$TVDB_PLUGIN_SHA256" ]; then
+        rm -f "$tmp"
+        die "artifact sha256 is $got, pinned $TVDB_PLUGIN_SHA256 — refusing to install"
+    fi
+    read -r guid version <<<"$(plugin_meta "$tmp")"
+    if [ "$guid" != "$TVDB_PLUGIN_GUID" ]; then
+        rm -f "$tmp"
+        die "artifact guid is $guid, expected the official $TVDB_PLUGIN_GUID — refusing to install"
+    fi
+    if [ "$version" != "$TVDB_PLUGIN_VERSION" ]; then
+        rm -f "$tmp"
+        die "artifact version is $version, pinned $TVDB_PLUGIN_VERSION — refusing to install"
+    fi
+    mkdir -p "$PLUGINS/$PLUGIN_DIR"
+    python3 -m zipfile -e "$tmp" "$PLUGINS/$PLUGIN_DIR/"
+    rm -f "$tmp"
+    [ -f "$PLUGINS/$PLUGIN_DIR/$TVDB_DLL" ] ||
+        die "$PLUGIN_DIR has no $TVDB_DLL — the artifact does not match this plugin"
+    printf '+ installed %s\n' "$PLUGIN_DIR"
+}
+
+backup_to_trash() {
+    local f="$1"
+    mkdir -p "$TRASH_DIR"
+    local dest="$TRASH_DIR/$(basename "$f").$(date +%H%M%S)"
+    cp -p "$f" "$dest"
+    printf '+ previous copy kept at %s\n' "$dest"
+}
+
+cmd_write_keys() {
+    local now want
+    KEYS_CHANGED=0
+    # TMDb: the running container owns this file and every setting in it. Only
+    # the key element is touched, so country, language and adult settings
+    # survive the edit.
+    [ -f "$TMDB_XML" ] ||
+        die "$TMDB_XML is missing — save the TMDb plugin settings once in the UI, then re-run"
+    now="$(xml_value "$TMDB_XML" TmdbApiKey)"
+    keys_len TMDB_API_KEY >/dev/null
+    want="$(keys_value TMDB_API_KEY)"
+    if [ "$now" = "$want" ]; then
+        printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
+    else
+        backup_to_trash "$TMDB_XML"
+        xml_secret "$TMDB_XML" TmdbApiKey TMDB_API_KEY
+        printf '+ set TmdbApiKey (%s chars)\n' "${#want}"
+        KEYS_CHANGED=1
+    fi
+
+    # TVDB: never overwrite a config the operator saved through the UI.
+    if [ -f "$TVDB_XML" ]; then
+        now="$(xml_value "$TVDB_XML" ApiKey)"
+        if [ -n "$now" ]; then
+            printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
+        else
+            keys_len TVDB_API_KEY >/dev/null
+            backup_to_trash "$TVDB_XML"
+            xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
+            if [ -n "$(keys_value TVDB_PIN)" ] && [ -z "$(xml_value "$TVDB_XML" Pin)" ]; then
+                xml_secret "$TVDB_XML" Pin TVDB_PIN
+            fi
+            printf '+ filled the ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
+            KEYS_CHANGED=1
+        fi
+    else
+        keys_len TVDB_API_KEY >/dev/null
+        cat >"$TVDB_XML" <<XML
+<?xml version="1.0" encoding="utf-8"?>
+<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <ApiKey />
+</Configuration>
+XML
+        xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
+        if [ -n "$(keys_value TVDB_PIN)" ]; then
+            xml_secret "$TVDB_XML" Pin TVDB_PIN
+        fi
+        printf '+ created Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
+        KEYS_CHANGED=1
+    fi
+}
+
+cmd_plan() {
+    check_root
+    check_keys_report
+    printf 'root  %s\n' "$JF_ROOT"
+    plugin_state
+    tmdb_state
+    tvdb_state
+}
+
+cmd_install() {
+    check_root
+    check_keys_report
+    KEYS_CHANGED=0
+    if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        write_plugin
+        KEYS_CHANGED=1
+    else
+        printf '= %s already installed\n' "$PLUGIN_DIR"
+    fi
+    cmd_write_keys
+    if [ "$KEYS_CHANGED" -eq 1 ]; then
+        # Nothing is restarted here on purpose: this host may be playing media
+        # right now, and both the plugin and its keys are read at start-up.
+        printf 'next: scripts/deploy.sh dankserver --service jellyfin (the plugin and keys load at start-up)\n'
+    else
+        printf '= no change\n'
+    fi
+}
+
+
+# A library whose options.xml carries EnableInternetProviders=false, or an empty
+# ImageFetchers list beside a populated MetadataFetchers list, is a SILENT FILE
+# INDEX: it indexes every file, answers 204 to every refresh, stores no provider
+# id, plot, rating or artwork, and logs nothing — so the item count reads like a
+# healthy scan. Jellyfin 12 keeps these options in root/default/<Library>/options.xml
+# (it was library.xml) and stores NONE of them in the database, so this audit is
+# the only place the state can be read without a browser session.
+verify_library_options() {
+    local root="$JF_ROOT/root/default" f lib problems
+    if [ ! -d "$root" ]; then
+        printf '= no library options under %s (nothing audited)\n' "$root"
+        return 0
+    fi
+    local rc=0
+    for f in "$root"/*/options.xml; do
+        [ -f "$f" ] || continue
+        lib="$(basename "$(dirname "$f")")"
+        problems="$(python3 -s - "$f" <<'PY'
+import sys, xml.dom.minidom as X
+root = X.parse(sys.argv[1]).documentElement
+def child(el, name):
+    for n in el.childNodes:
+        if n.nodeType == n.ELEMENT_NODE and n.tagName == name:
+            return n
+def flag(el, name):
+    n = child(el, name)
+    return None if n is None else (n.firstChild.data.strip() if n.firstChild else "")
+def names(el, name):
+    n = child(el, name)
+    return None if n is None else [s.firstChild.data.strip() for s in n.childNodes
+                                   if s.nodeType == s.ELEMENT_NODE and s.firstChild]
+bad = []
+ip = flag(root, "EnableInternetProviders")
+if ip is not None and ip.lower() != "true":
+    bad.append("EnableInternetProviders=%s" % (ip or "empty"))
+types = child(root, "TypeOptions")
+if types is not None:
+    for t in types.childNodes:
+        if t.nodeType != t.ELEMENT_NODE or t.tagName != "TypeOptions":
+            continue
+        meta = names(t, "MetadataFetchers") or []
+        if meta and not (names(t, "ImageFetchers") or []):
+            bad.append("%s: ImageFetchers empty while MetadataFetchers names %d"
+                       % (flag(t, "Type") or "?", len(meta)))
+print("; ".join(bad))
+PY
+)"
+        if [ -n "$problems" ]; then
+            printf 'SILENT INDEX %s: %s\n' "$lib" "$problems" >&2
+            rc=1
+        else
+            printf '= %s: library options fetch from providers\n' "$lib"
+        fi
+    done
+    return "$rc"
+}
+
+cmd_verify() {
+    check_root
+    local bad=0
+    if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        printf 'MISSING plugin directory %s (thetvdb not installed)\n' "$PLUGIN_DIR" >&2
+        bad=1
+    else
+        local dlls abi
+        dlls="$(find "$PLUGINS/$PLUGIN_DIR" -maxdepth 1 -name '*.dll' | wc -l)"
+        [ "$dlls" -ge 1 ] || {
+            printf 'MISSING %s inside %s\n' "$TVDB_DLL" "$PLUGIN_DIR" >&2
+            bad=1
+        }
+        abi="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("targetAbi","?"))' \
+            "$PLUGINS/$PLUGIN_DIR/meta.json" 2>/dev/null || echo '?')"
+        printf 'plugin %s: %s dll(s), targetAbi %s\n' "$PLUGIN_DIR" "$dlls" "$abi"
+    fi
+    local pair elem n
+    for pair in "$TMDB_XML|TmdbApiKey" "$TVDB_XML|ApiKey"; do
+        elem="${pair##*|}"
+        if [ ! -f "${pair%%|*}" ]; then
+            printf 'MISSING %s\n' "${pair%%|*}" >&2
+            bad=1
+            continue
+        fi
+        if ! python3 -c 'import sys, xml.dom.minidom; xml.dom.minidom.parse(sys.argv[1])' "${pair%%|*}"; then
+            printf 'UNPARSEABLE %s\n' "${pair%%|*}" >&2
+            bad=1
+            continue
+        fi
+        n="$(xml_value "${pair%%|*}" "$elem")"
+        if [ -z "$n" ]; then
+            printf 'MISSING key: %s has no %s value\n' "$(basename "${pair%%|*}")" "$elem" >&2
+            bad=1
+        else
+            printf '%s %s chars\n' "$elem" "${#n}"
+        fi
+    done
+    verify_library_options || bad=1
+    [ "$bad" -eq 0 ] || return 1
+    printf 'providers configured\n'
+}
+
+cmd="${1:-}"
+case "$cmd" in
+plan) cmd_plan ;;
+install) cmd_install ;;
+verify) cmd_verify ;;
+*) usage ;;
+esac

scripts/media-ab-reorg.py

@@ -0,0 +1,420 @@
+#!/usr/bin/env python3
+"""media-ab-reorg.py — plan the Audiobooks dedupe + ABS-structure relayout.
+
+2026-11-04 campaign. Turns the Audiobooks library into the layout Audiobookshelf
+documents (https://audiobookshelf.org/docs/documentation/libraries/book-library/
+directory-structure/):
+
+    {Author}/[{Series}/]{seq} - {year} - {Title} {Narrator}/…files…
+
+The authority for every book's author/series/title/year/narrator is the ABS
+database dump (--items, JSON lines: [relPath, isMissing, title, authors, series,
+seq, year, narratorsJSON, coverPath, ebookFlag]); files are attached to their
+book by longest-prefix match on the item's relPath, which is how ABS's own
+grouping works. Folder names are then re-emitted so ABS's folder parser reads
+the same metadata back off disk.
+
+Three outputs, all consumable by scripts/media-apply.py (run everything with
+--root /mnt/entropy; hold/intake/park live outside the library on purpose):
+
+  --plan        mkdir/keep/move rows: dedupe losers -> --hold, ebook-only items
+                and comics -> --intake (the eBooks pile feeds Calibre next),
+                unowned media -> --park, everything else -> the new layout.
+  --covers      rows "srchost \\t dstarray": covers ABS keeps in /metadata get a
+                copy beside the book (cp, it crosses filesystems); run this
+                BEFORE apply so the cover lands inside the moved folder.
+  --exceptions  every judgement call: name collisions, byte-dups that pair two
+                different books, collapsed items, stray files.
+
+Dedupe law: only CONFIRMED byte-identities (--dups from
+media-ab-verifydups.py) remove a file. One copy survives per book: inside a
+folder the cleanest-named file wins; across folders the item whose root matches
+its author wins. Equal bytes under two different book identities keep both and
+raise an exception — a trailer embedded in two books is not a duplicate.
+
+Rerunnable: a second run against a moved library emits keep rows, not moves.
+"""
+import argparse
+import json
+import os
+import re
+from collections import defaultdict
+
+AUDIO = {"m4b", "mp3", "m4a", "flac", "opus", "ogg", "oga", "mp4", "aac", "wma",
+         "aiff", "aif", "wav", "webm", "webma", "mka", "awb", "caf", "mpg",
+         "mpg2", "mpeg"}
+EBOOK = {"epub", "pdf", "mobi", "azw3", "cbr", "cbz", "fb2"}
+COMIC = {"cbr", "cbz"}
+MEDIA = AUDIO | {"epub", "pdf", "mobi", "azw3", "cbr", "cbz"}
+PLACEHOLDER_AUTHOR = {"unknown", "unknown artist", "various", "various artists",
+                      "various authors", "n/a", ""}
+BAD_CHARS = re.compile(r'[<>:"/\\|?*]')
+DISC_DIR = re.compile(r"^(disc|cd|disk)\s*\d", re.I)
+KEEP_HINT = re.compile(r"\s*\((?:2|3|copy|backup|duplicate)\)\s*$", re.I)
+META_HOST = "/home/dank/Containers/audiobookshelf"
+
+
+def clean(name):
+    name = BAD_CHARS.sub(" ", name or "")
+    name = re.sub(r"\s+", " ", name).strip().strip(".")
+    return name or "Untitled"
+
+
+def clean_author(name):
+    """DB author string -> folder name: one author, no 'zz' sort artifact."""
+    name = re.sub(r"^zz", "", (name or "").strip())
+    return clean(name)
+
+
+def first_author(authors):
+    parts = [clean_author(p) for p in (authors or "").split("|")]
+    parts = [p for p in parts if p]
+    return parts[0] if parts else "Unknown"
+
+
+def parse_narrators(blob):
+    try:
+        v = json.loads(blob) if blob else []
+    except (TypeError, json.JSONDecodeError):
+        v = []
+    if isinstance(v, str):
+        v = [v]
+    return [str(x).strip() for x in (v or []) if str(x).strip()]
+
+
+def core_title(title, author):
+    """Title as the book folder should show it: release prefixes stripped."""
+    t = (title or "Untitled").strip()
+    a = (author or "").strip()
+    if a and t.lower().startswith(a.lower() + " -"):
+        t = t[len(a):].lstrip(" -")
+    return t.strip() or "Untitled"
+
+
+def bookname(seq, year, title, narrators):
+    seq = (seq or "").strip()
+    seq = seq if re.fullmatch(r"\d+(\.\d+)?", seq) else ""
+    year = (year or "").strip()
+    year = year if re.fullmatch(r"(1[5-9]|20)\d\d", year) else ""
+    t = title
+    if seq:
+        # drop a sequence prefix already carried inside the title
+        t = re.sub(rf"^{re.escape(seq)}\s*[\-.]\s+", "", t)
+    if year:
+        t = re.sub(rf"^{year}\s*[\-.]\s+", "", t)
+        t = re.sub(rf"\s*\({year}\)\s*$", "", t)
+    name = t
+    if seq and year:
+        name = f"{seq} - {year} - {name}"
+    elif seq:
+        name = f"{seq} - {name}"
+    elif year:
+        name = f"{year} - {name}"
+    if len(narrators) == 1:
+        name = f"{name} {{{clean(narrators[0])}}}"
+    return clean(name)
+
+
+class Campaign:
+    def __init__(self, args):
+        self.root = args.root.rstrip("/")
+        self.lib = self.root + "/Audiobooks"
+        self.hold = args.hold
+        self.intake = args.intake
+        self.park = args.park
+        self.items = {}          # relPath -> item dict (healthy only)
+        self.stale = []          # isMissing=1 rows
+        self.moved = {}          # src -> (dst, kind)
+        self.claims = {}         # dst -> src (collision guard)
+        self.loser_files = set()
+        self.loser_items = set()
+        self.exc = defaultdict(list)
+
+    # ------------------------------------------------------------- inputs
+    def load_items(self, path):
+        for line in open(path):
+            r = json.loads(line)
+            rel = r[0]
+            item = {"rel": rel, "title": r[2] or "", "authors": r[3] or "",
+                    "series": r[4] or "", "seq": r[5] or "", "year": r[6] or "",
+                    "narr": parse_narrators(r[7]), "cover": r[8] or "",
+                    "ebook": r[9]}
+            if r[1]:
+                self.stale.append(rel)
+                continue
+            self.items[rel] = item
+
+    def load_files(self, path):
+        files = []
+        for line in open(path):
+            sz, ext, p = line.rstrip("\n").split("\t", 2)
+            if p.startswith(self.lib + "/"):
+                files.append((int(sz), ext.lstrip(".").lower(), p))
+        return files
+
+    def owner(self, d):
+        """The healthy DB item whose folder is d or an ancestor of d."""
+        rel = os.path.relpath(d, self.lib)
+        while rel and rel != ".":
+            if rel in self.items:
+                return rel
+            rel = os.path.dirname(rel)
+        return None
+
+    # ------------------------------------------------------------- dedupe
+    def identity(self, item_rel):
+        if item_rel is None:
+            return None
+        it = self.items[item_rel]
+        return (first_author(it["authors"]).lower(),
+                core_title(it["title"], first_author(it["authors"])).lower(),
+                (it["year"] or "").strip())
+
+    def root_score(self, item_rel):
+        it = self.items[item_rel]
+        top = item_rel.split("/")[0]
+        s = 0
+        if top.lower() == first_author(it["authors"]).lower():
+            s += 2
+        if not top.startswith("_") and "various" not in top.lower():
+            s += 1
+        return (s, -len(item_rel))
+
+    def dedupe(self, files, dups_path):
+        by_hash = defaultdict(list)
+        if dups_path:
+            for line in open(dups_path):
+                h, sz, p = line.rstrip("\n").split("\t", 2)
+                if p.startswith(self.lib + "/"):
+                    by_hash[h].append(p)
+        owner_of = {}
+        dirs = {os.path.dirname(p) for _, _, p in files}
+        for d in dirs:
+            owner_of[d] = self.owner(d)
+        files_by_owner = defaultdict(list)
+        for _, _, p in files:
+            files_by_owner[owner_of[os.path.dirname(p)]].append(p)
+
+        for h, paths in by_hash.items():
+            owners = {owner_of.get(os.path.dirname(p)) for p in paths}
+            owners = {o for o in owners if o is not None}
+            per_item = defaultdict(list)
+            for p in paths:
+                per_item[owner_of.get(os.path.dirname(p))].append(p)
+            if owners:
+                ids = {o: self.identity(o) for o in owners}
+                # group items by identity; loose (None) files follow their own rule
+                same_as_loser = []
+                keepers = []
+                byid = defaultdict(list)
+                for o in owners:
+                    byid[ids[o]].append(o)
+                for ident, os_ in byid.items():
+                    os_.sort(key=self.root_score, reverse=True)
+                    keepers.append(os_[0])
+                    same_as_loser.extend(os_[1:])
+                keepers = set(keepers)
+                loser_items = set(same_as_loser)
+            else:
+                loser_items = set()
+                keepers = set()
+            for o, ps in per_item.items():
+                if o is None:
+                    continue  # loose files decided in placement
+                if o in loser_items:
+                    self.loser_files.update(ps)
+                    continue
+                if len(ps) > 1 and o in keepers:
+                    # redundant copies inside the surviving folder
+                    keeper = min(ps, key=lambda p: (
+                        bool(KEEP_HINT.search(os.path.splitext(p)[0])),
+                        len(p), p))
+                    self.loser_files.update(p for p in ps if p != keeper)
+            for o in loser_items:
+                self.loser_items.add(o)
+        # collapsed items: every media file lost
+        for o, ps in files_by_owner.items():
+            if o is None:
+                continue
+            media = [p for p in ps if os.path.splitext(p)[1].lstrip(".").lower()
+                     in MEDIA]
+            if media and all(p in self.loser_files for p in media):
+                self.loser_items.add(o)
+                self.exc["collapsed items (all files were byte-duplicates)"].append(o)
+
+    # ---------------------------------------------------------- placement
+    def target(self, item):
+        author = first_author(item["authors"])
+        series = clean(item["series"]) if item["series"] else ""
+        title = core_title(item["title"], author)
+        name = bookname(item["seq"], item["year"], title, item["narr"])
+        parts = [self.lib, clean(author)]
+        if series:
+            parts.append(clean(series))
+        parts.append(name)
+        return os.path.join(*parts)
+
+    def claim(self, dst, src, why):
+        """One destination, one source; collisions rename with a marker."""
+        if dst not in self.claims:
+            self.claims[dst] = src
+            return dst
+        stem, ext = os.path.splitext(dst)
+        n = 2
+        while f"{stem} ({n}){ext}" in self.claims:
+            n += 1
+        dst2 = f"{stem} ({n}){ext}"
+        self.claims[dst2] = src
+        self.exc[why].append(f"{src} -> renamed to {dst2}")
+        return dst2
+
+    def place(self, files):
+        audio_by_item = defaultdict(int)
+        for _, ext, p in files:
+            o = self.owner(os.path.dirname(p))
+            if o and ext in AUDIO:
+                audio_by_item[o] += 1
+        cover_dirs = {}
+        # Claim one BOOK FOLDER per item: two items for the same title (the
+        # _unsorted era filed some books 3 times) must not merge their tracks
+        # into one item — the loser gets a variant folder instead.
+        for rel in sorted(self.items, key=lambda r: self.root_score(r),
+                          reverse=True):
+            it = self.items[rel]
+            t = self.target(it)
+            if t in self.claims:
+                n = 2
+                while f"{t} (variant {n})" in self.claims:
+                    n += 1
+                t = f"{t} (variant {n})"
+                self.exc["same title filed by two DB items"].append(f"{rel} -> {t}")
+            self.claims[t] = rel
+            cover_dirs[rel] = t
+            if (it["cover"].startswith("/metadata") and audio_by_item[rel]
+                    and rel not in self.loser_items):
+                src = META_HOST + it["cover"]
+                self.covers.append((src, os.path.join(t,
+                                                      "cover" + os.path.splitext(src)[1].lower())))
+        # Containment gate: an item folder holding media files must never sit
+        # above another item folder — ABS's grouping would swallow the child
+        # (the docs' PROBLEM example). Rename the ANCESTOR book out of the way.
+        for _ in range(3):
+            moved = False
+            folders = sorted(set(cover_dirs.values()))
+            for anc in folders:
+                for d in folders:
+                    if d != anc and d.startswith(anc + os.sep):
+                        n = 1
+                        cand = f"{anc} (book)" if n == 1 else f"{anc} (book {n})"
+                        while cand in self.claims:
+                            n += 1
+                            cand = f"{anc} (book {n})"
+                        self.exc["book folder nested above another book"].append(
+                            f"{anc} -> {cand}")
+                        self.claims[cand] = self.claims.pop(anc)
+                        for r, t in cover_dirs.items():
+                            if t == anc:
+                                cover_dirs[r] = cand
+                        self.covers = [(s, d.replace(anc + os.sep, cand + os.sep, 1))
+                                       for s, d in self.covers]
+                        moved = True
+                        break
+                if moved:
+                    break
+            if not moved:
+                break
+        for _, ext, p in files:
+            if p in self.loser_files:
+                d = os.path.join(self.hold, os.path.relpath(p, self.root))
+                self.move(p, d, "dup")
+                continue
+            d = os.path.dirname(p)
+            o = self.owner(d)
+            if o is None:
+                rel = os.path.relpath(p, self.lib)
+                if ext in MEDIA:
+                    self.move(p, os.path.join(self.park, rel), "park")
+                else:
+                    self.move(p, os.path.join(self.park, rel), "park-junk")
+                continue
+            it = self.items[o]
+            if ext in COMIC:
+                key = clean(f"{first_author(it['authors'])} - {core_title(it['title'], '')}")
+                dst = os.path.join(self.intake, "_comics", key, os.path.basename(p))
+                self.move(p, self.claim(dst, p, "intake comic collisions"), "comic")
+                continue
+            if audio_by_item[o] == 0:  # ebook-only item belongs to the ebook stack
+                key = f"{first_author(it['authors'])} - {bookname(it['seq'], it['year'], core_title(it['title'], first_author(it['authors'])), it['narr'])}"
+                dst = os.path.join(self.intake, key, os.path.relpath(p, os.path.join(self.lib, o)))
+                self.move(p, self.claim(dst, p, "intake ebook collisions"), "ebook-only")
+                continue
+            rel_in = os.path.relpath(p, os.path.join(self.lib, o))
+            dst = os.path.join(cover_dirs[o], rel_in)
+            if dst == p:
+                self.kept.append(p)
+                self.claims.setdefault(dst, p)
+                continue
+            self.move(p, self.claim(dst, p, "target collisions"), "move")
+
+    def move(self, src, dst, kind):
+        self.moves.append((src, dst, kind))
+
+    def write(self, plan_path, exc_path):
+        dirs = {os.path.dirname(dst) for _, dst, _ in self.moves}
+        dirs |= {os.path.dirname(d) for _, d in self.covers}
+        all_dirs = set()
+        for d in dirs:
+            while d and d != self.root:
+                all_dirs.add(d)
+                d = os.path.dirname(d)
+        with open(plan_path, "w") as fh:
+            fh.write("action\tsrc\tdst\tkind\n")
+            for d in sorted(all_dirs):
+                fh.write(f"mkdir\t\t{d}\tcampaign\n")
+            for p in self.kept:
+                fh.write(f"keep\t{p}\t\tkeep\n")
+            for src, dst, kind in sorted(self.moves):
+                fh.write(f"move\t{src}\t{dst}\t{kind}\n")
+        if self.exc:
+            with open(exc_path, "w") as fh:
+                for why in sorted(self.exc):
+                    fh.write(f"# {why}\n")
+                    for line in sorted(set(self.exc[why])):
+                        fh.write(line + "\n")
+                    fh.write("\n")
+        mv = [m for m in self.moves if m[2] != "dup"]
+        print(f"plan: {len(all_dirs)} folders, {len(self.moves)} moves "
+              f"({len(mv)} relayout, {len(self.moves)-len(mv)} dup-hold), "
+              f"{len(self.kept)} keep, {len(self.covers)} cover-copies")
+
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default="/mnt/entropy")
+    ap.add_argument("--enum", required=True)
+    ap.add_argument("--items", required=True)
+    ap.add_argument("--dups", default=None)
+    ap.add_argument("--hold", default="/mnt/entropy/.media-campaign/dup-hold-20261104")
+    ap.add_argument("--intake", default="/mnt/entropy/eBooks/_from-audiobooks")
+    ap.add_argument("--park", default="/mnt/entropy/NeedsProcessing/_from-audiobooks-20261104")
+    ap.add_argument("--plan", required=True)
+    ap.add_argument("--exceptions", required=True)
+    ap.add_argument("--covers", required=True)
+    args = ap.parse_args()
+
+    c = Campaign(args)
+    c.moves, c.kept, c.covers = [], [], []
+    c.load_items(args.items)
+    files = c.load_files(args.enum)
+    c.dedupe(files, args.dups)
+    c.place(files)
+    c.write(args.plan, args.exceptions)
+    with open(args.covers, "w") as fh:
+        for src, dst in sorted(set(c.covers)):
+            fh.write(f"{src}\t{dst}\n")
+    print(f"covers: {len(set(c.covers))}  stale DB rows (delete after scan): {len(c.stale)}")
+
+
+if __name__ == "__main__":
+    main()

scripts/media-ab-verifydups.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+"""media-ab-verifydups.py — confirm byte-duplicates among size-group survivors.
+
+Stage 2 of the 2026-11-04 Audiobooks dedupe. Stage 1 (partial blake2b of the
+first+last 4 MiB, done by the campaign's hashcand.py) over-claims: two files
+of equal size and equal head+tail are CANDIDATES, not copies. This tool takes
+the candidate files from --partial (rows: size, hash, path), groups them by
+(size, partial-hash), full-hashes only members of groups with 2+ members, and
+writes verdicts: one line per file whose full hash ties it to at least one
+other file.
+
+Output rows: full-hash <TAB> size <TAB> path   (sorted by hash) — a hash that
+appears once is not written at all; singletons are not duplicates.
+
+Read-only. Never moves, never deletes. Run it after the partial pass finishes
+(pgrep it first; two hashers on one output file have raced before).
+"""
+import argparse
+import hashlib
+import os
+from collections import defaultdict
+
+
+def full_hash(path, chunk=1 << 20):
+    h = hashlib.blake2b(digest_size=32)
+    with open(path, "rb") as fh:
+        for block in iter(lambda: fh.read(chunk), b""):
+            h.update(block)
+    return h.hexdigest()
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--partial", required=True, help="partial-hashes.tsv")
+    ap.add_argument("--out", required=True, help="full-hash verdicts TSV")
+    ap.add_argument("--progress-every", type=int, default=50)
+    args = ap.parse_args()
+
+    groups = defaultdict(list)
+    with open(args.partial) as fh:
+        for line in fh:
+            f = line.rstrip("\n").split("\t")
+            if len(f) != 3 or f[0] == "ERR":
+                continue
+            groups[(int(f[0]), f[1])].append(f[2])
+    survivors = sorted({p for g in groups.values() if len(g) > 1 for p in g})
+    print(f"survivor files to full-hash: {len(survivors)}")
+
+    done = set()
+    if os.path.exists(args.out):
+        with open(args.out) as fh:
+            for line in fh:
+                done.add(line.rstrip("\n").split("\t", 2)[2])
+    full = {}
+    for line in open(args.out) if os.path.exists(args.out) else []:
+        h, s, p = line.rstrip("\n").split("\t", 2)
+        full[p] = h
+
+    todo = [p for p in survivors if p not in done]
+    out = open(args.out, "a")
+    for i, p in enumerate(todo):
+        try:
+            full[p] = full_hash(p)
+        except OSError as e:
+            print(f"SKIP {p}: {e}")
+            continue
+        if i % args.progress_every == args.progress_every - 1:
+            out.flush()
+    out.close()
+
+    # Rewrite the whole verdict file keeping only TRUE duplicates (2+ fulls).
+    by_hash = defaultdict(list)
+    for p in survivors:
+        if p in full:
+            by_hash[full[p]].append(p)
+    with open(args.out, "w") as out:
+        for h, ps in sorted(by_hash.items()):
+            if len(ps) > 1:
+                for p in sorted(ps):
+                    out.write(f"{h}\t{os.path.getsize(p)}\t{p}\n")
+    dups = {h: ps for h, ps in by_hash.items() if len(ps) > 1}
+    n = sum(len(ps) - 1 for ps in dups.values())
+    waste = sum((len(ps) - 1) * os.path.getsize(ps[0]) for ps in dups.values())
+    print(f"true dup sets: {len(dups)}  redundant files: {n}  "
+          f"waste: {waste / 2**30:.1f} GiB")
+
+
+if __name__ == "__main__":
+    main()

scripts/media-apply.py

@@ -32,6 +32,12 @@ cross a filesystem (rename cannot do it and a silent copy at 24 TB would not).
 --force relaxes only the guards that are opinion (journal already rolled back,
 paths outside --root); it never allows overwriting a file.
 
+--metadata-view names the overlay the media servers read their library through
+(default /var/lib/media-meta/view). The sidecar follow-up printed after an apply
+or rollback is conditioned on that path being a mountpoint: where no view is
+mounted the servers bind the array itself, the sidecars moved with the media,
+and a reap order would point at a directory that is not there.
+
 The journal defaults to `media-relayout.jsonl` BESIDE the library root, never
 inside it: a log inside the library is a file Jellyfin will index.
 """
@@ -53,10 +59,18 @@ def journal_default(root):
 
 
 class Journal:
-    """Append-only JSONL of intents. Writes are fsynced before the tool acts."""
+    """Append-only JSONL of intents. Writes are fsynced before the tool acts.
 
-    def __init__(self, path, writing):
+    ``batch`` is how many intents may be written before one fsync covers them.
+    The rule the file exists for is unchanged — an intent is on disk before the
+    move it authorises runs — so the caller must call ``sync()`` before it
+    performs a batch's renames; only the number of flushes falls. 1 = a flush
+    per intent (the original behaviour).
+    """
+
+    def __init__(self, path, writing, batch=1):
         self.path, self.writing = path, writing
+        self.batch = max(1, int(batch))
         self.records = []
         if os.path.exists(path):
             for line in open(path):
@@ -64,11 +78,15 @@ class Journal:
                 if line:
                     self.records.append(json.loads(line))
         self._fh = None
+        self._unsynced = 0
+        # (src, dst) index: already() is called three times per move, so an
+        # any() over every record written so far made a 58k-move pass quadratic.
+        self._intents = {(r.get("src"), r.get("dst"))
+                         for r in self.records if r.get("op") == "move"}
 
     def already(self, dst, src):
         """An intent for this exact (src, dst) was written by an earlier run."""
-        return any(r.get("op") == "move" and r.get("dst") == dst
-                   and r.get("src") == src for r in self.records)
+        return (src, dst) in self._intents
 
     @property
     def rolled_back(self):
@@ -76,29 +94,41 @@ class Journal:
 
     def record(self, **fields):
         self.records.append(fields)
+        if fields.get("op") == "move":
+            self._intents.add((fields.get("src"), fields.get("dst")))
         if not self.writing:
             return
         if self._fh is None:
             self._fh = open(self.path, "a")
         self._fh.write(json.dumps(fields) + "\n")
         self._fh.flush()
+        self._unsynced += 1
+        if self._unsynced >= self.batch:
+            self.sync()
+
+    def sync(self):
+        """Make every intent written so far durable. No-op when nothing pending."""
+        if not self.writing or self._fh is None or not self._unsynced:
+            return
         os.fsync(self._fh.fileno())
+        self._unsynced = 0
 
     def close(self):
         if self._fh:
+            self.sync()
             self._fh.close()
 
 
 def read_plan(path, root, force):
     """Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root."""
-    mkdirs, keeps, moves = [], [], []
+    mkdirs, keeps, moves, bad = [], [], [], []
     with open(path) as fh:
         header = fh.readline()
         if not header.startswith("action\t"):
             raise Refuse(f"{path} is not a plan (first column is "
                          f"{header.split(chr(9))[0]!r}, expected 'action') —"
                          " regenerate it with scripts/media-plan.py")
-        for line in fh:
+        for n, line in enumerate(fh, 2):
             f = line.rstrip("\n").split("\t")
             if f[0] == "mkdir":
                 mkdirs.append(f[2])
@@ -106,6 +136,14 @@ def read_plan(path, root, force):
                 keeps.append(f[1])
             elif f[0] == "move":
                 moves.append((f[1], f[2], f[3] if len(f) > 3 else ""))
+            else:
+                bad.append(f"line {n}: unknown action {f[0]!r} — this tool"
+                           " knows mkdir, keep and move (a rename is a move)")
+    if bad:
+        # A row this tool could not read is a plan bug, and a plan bug refuses:
+        # silently dropping it turns a typo into a no-op run that reports success.
+        head = "; ".join(bad[:3])
+        raise Refuse(head + (f" (+{len(bad) - 3} more)" if len(bad) > 3 else ""))
     if not force:
         for p in mkdirs + keeps + [p for m in moves for p in m[:2]]:
             if p and not (p == root or p.startswith(root + os.sep)):
@@ -114,6 +152,24 @@ def read_plan(path, root, force):
     return mkdirs, keeps, moves
 
 
+# The media servers may read the library through the SSD overlay view
+# (scripts/media-view.sh). A relayout moves bytes on the raw array, so sidecars
+# the servers wrote through that view stay at the old coordinates — naming the
+# follow-up here is the point of use for that fact, but only where the view
+# exists: where it is not mounted the servers bind the array itself, the
+# sidecars moved with the media, and a reap order points at a directory that
+# is not there. See the Media Metadata View page for the host's state.
+METADATA_VIEW = "/var/lib/media-meta/view"
+METADATA_HINT = ("note: sidecars written through the metadata view did not follow"
+                 " these moves — next: scripts/media-view.sh reap")
+
+
+def metadata_note(args):
+    """Print the sidecar follow-up only when the overlay is really mounted."""
+    if os.path.ismount(args.metadata_view):
+        print(METADATA_HINT)
+
+
 def do_apply(args, root, journal, mkdirs, keeps, moves):
     """Preflight everything, then folders, then renames. Never a partial start."""
     problems = []
@@ -153,14 +209,30 @@ def do_apply(args, root, journal, mkdirs, keeps, moves):
             print(f"  would move {src} -> {dst}")
         return 0
 
+    batch = max(1, args.batch)
     made = {r.get("dst") for r in journal.records if r.get("op") == "mkdir"}
     dirs_new = 0
+    queued = []
+
+    def drain_dirs():
+        """Create the queued folders only after their intents are durable."""
+        nonlocal dirs_new
+        if not queued:
+            return
+        journal.sync()
+        for d in queued:
+            os.makedirs(d, exist_ok=True)
+            dirs_new += 1
+        queued.clear()
+
     for d in sorted(mkdirs, key=lambda q: q.count(os.sep)):
         if d in made or os.path.isdir(d):
             continue
         journal.record(op="mkdir", dst=d)
-        os.makedirs(d, exist_ok=True)
-        dirs_new += 1
+        queued.append(d)
+        if len(queued) >= batch:
+            drain_dirs()
+    drain_dirs()
 
     # same-filesystem gate: rename cannot cross a device, and a silent copy of a
     # library this size is not an option anyone would choose.
@@ -177,6 +249,19 @@ def do_apply(args, root, journal, mkdirs, keeps, moves):
         return 2
 
     moved = done = 0
+    pending = []
+
+    def drain_moves():
+        """Perform the queued renames only after their intents are on disk."""
+        nonlocal moved
+        if not pending:
+            return
+        journal.sync()
+        for s, d in pending:
+            os.rename(s, d)
+            moved += 1
+        pending.clear()
+
     for src, dst, _kind in moves:
         if journal.already(dst, src):
             if os.path.exists(dst) and not os.path.exists(src):
@@ -189,12 +274,15 @@ def do_apply(args, root, journal, mkdirs, keeps, moves):
         st = os.stat(src)
         journal.record(op="move", src=src, dst=dst, bytes=st.st_size, mode=st.st_mode,
                        mtime=st.st_mtime, ino=st.st_ino, dev=st.st_dev)
-        os.rename(src, dst)
-        moved += 1
+        pending.append((src, dst))
+        if len(pending) >= batch:
+            drain_moves()
+    drain_moves()
     print(f"applied: {moved} moved, {done} already done in an earlier run,"
           f" {dirs_new} folders created, {len(keeps)} files stayed put")
     print(f"journal: {journal.path} — undo with: media-apply.py rollback"
           f" --journal {journal.path} --root {root} --apply")
+    metadata_note(args)
     return 0
 
 
@@ -378,6 +466,7 @@ def do_rollback(args, root, journal):
                    t=datetime.datetime.now(datetime.timezone.utc).isoformat())
     print(f"rolled back: {undone} files back where the census found them,"
           f" {dirs_gone} empty folders removed, {recreated} swept folders recreated")
+    metadata_note(args)
     return 0
 
 
@@ -389,13 +478,20 @@ def main():
     ap.add_argument("--journal")
     ap.add_argument("--apply", action="store_true", help="actually change the library")
     ap.add_argument("--force", action="store_true", help="relax the opinion guards")
+    ap.add_argument("--metadata-view", default=METADATA_VIEW,
+                    help="overlay the media servers read through; the sidecar"
+                         " note prints only when this path is a mountpoint")
+    ap.add_argument("--batch", type=int, default=64,
+                    help="journal intents per fsync (1 = flush before every move)."
+                         " An intent is still durable before the move it authorises;"
+                         " only the number of flushes falls")
     args = ap.parse_args()
 
     root = os.path.abspath(args.root)
     if not os.path.isdir(root):
         raise Refuse(f"no library at {root}")
     journal_path = args.journal or journal_default(root)
-    journal = Journal(journal_path, args.apply)
+    journal = Journal(journal_path, args.apply, batch=args.batch)
     try:
         if args.cmd in ("rollback", "verify", "sweep") and not os.path.exists(journal_path):
             raise Refuse(f"no journal at {journal_path} — nothing to {args.cmd}")

scripts/media-audiobooks.py

@@ -0,0 +1,425 @@
+#!/usr/bin/env python3
+r"""media-audiobooks.py — a plan that re-files the audiobook library into one folder per book.
+
+    media-audiobooks.py --root DIR --enum TSV --rows TSV [--out TSV] [--exceptions FILE]
+
+Plex reads this library and Jellyfin does not: its music scanner has no audiobook
+concept, and the tree is indexed entirely by Plex, which matched 58,975 of the
+62,879 files here into its Artist/Album/Track shape. So the authority is
+`scripts/media-census.py audiobooks` — the BOOK (album row), its NARRATOR (that
+book's single artist row) and its CHAPTERS (track rows) — and never the folder a
+release pack made. The plan is:
+
+    Audiobooks/<Narrator>/<Book> (<year>)/<the file's own name>.<ext>
+                     ... the same folder, " (2)" on the name, when two chapters
+                         of one book carry the same name
+
+What that buys is the one property both media servers' book scanners assume: a
+folder holds exactly one book. Today 212 top-level folders are a mix of
+author-named shelves and collection packs, and the books inside the packs are
+split across `… [Part 1]`, `… [Part 2]` folders three or four deep.
+
+Three laws here are the opposite of what a music shelf does, and each is forced
+by what the rows actually hold:
+
+ * A chapter is NEVER renumbered. 7,556 of the 10,485 books contain at least one
+   chapter whose index Plex does not know, so a number written into a name would
+   be invented — and it would buy nothing, because both servers order playback by
+   the embedded track tag, not by the file name. A wrong number in a name is worse
+   than none: it is the number a human later trusts. A file already carrying its
+   own number keeps it, untouched.
+ * The shelf is the NARRATOR, not the author. Every book has exactly one artist row
+   and it is the narrator; the author survives only inside 10% of the album titles
+   (`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving the
+   author would mean parsing titles for one book in ten and inventing it for the
+   rest. One rule that always resolves beats a shelf that is half author and half
+   narrator; the author stays visible in the book folder's own name, which is the
+   title Plex shows.
+ * Nothing is guessed at. A chapter row with no book or no narrator parks — a
+   tag-scan placeholder (`[Unknown Album]`) names neither, since unrelated books
+   share it and filing on one merges them, and so does a chapter number the ripper
+   mistagged as the artist (`0.4`). An audio file Plex never matched parks, a
+   video container parks, and a loose file in a folder holding two books parks.
+
+Nothing is deleted and nothing is overwritten. The output is a list of rename(2)
+intents for `scripts/media-apply.py`, whose preflight refuses any target that is
+already claimed and whose journal is the rollback.
+"""
+import argparse
+import csv
+import os
+import re
+import sys
+from collections import Counter, defaultdict
+
+# What a chapter file can be. .m4b is the audiobook container; the rest are the
+# audio a pack ships when it was ripped as music.
+AUDIO_EXT = {".mp3", ".m4a", ".m4b", ".ogg", ".oga", ".opus", ".wav", ".flac",
+             ".wma", ".aiff", ".aif", ".m4v", ".aac", ".mka"}
+# Never an audiobook, whatever the library it sits in says: Plex reads these as
+# video, and renaming one into an audio extension would claim it is a chapter.
+VIDEO_EXT = {".mkv", ".mp4", ".webm", ".avi", ".mov", ".mpg", ".mpeg", ".ts",
+             ".m2ts", ".vob", ".ifo", ".wmv", ".iso", ".img", ".bdmv", ".ogm",
+             ".divx"}
+
+# Tag-scan placeholders. These strings are not identities: 1,106 chapters from
+# 14 unrelated top-level folders carry the album `[Unknown Album]`, so filing on
+# them would merge books that have nothing to do with each other — the exact
+# shape this relayout exists to make impossible. Such a chapter parks with the
+# path it has today and the fix belongs in its tags. `Various Artists` is NOT a
+# placeholder: that is what a multi-cast read carries, and the one book here
+# with 1,189 chapters under it is genuinely one book.
+PLACEHOLDER_BOOK = {"[unknown album]", "unknown album", "no title", "untitled",
+                    "[untitled]", "unknown title", "n/a", "-"}
+PLACEHOLDER_NARRATOR = {"[unknown artist]", "unknown artist", "unknown author",
+                        "<unknown>", "[unknown]", "unknown"}
+
+
+def identity(value, placeholders, needs_letter=False):
+    """The tag value Plex matched, when it names something; '' when it does not.
+
+    A trailing parenthesized group is stripped for the test only, because rippers
+    glue timestamps to the placeholder (`Unknown album (03/03/2015 21:39:55)`).
+
+    `needs_letter` is for a NARRATOR alone: a shelf is a person's name, and 272
+    chapters here carry a chapter number in the artist tag (`0.4`, `31`). The same
+    test must never judge a BOOK title, which may honestly be `1984`.
+    """
+    v = (value or "").strip()
+    bare = re.sub(r"\s*\([^()]*\)\s*$", "", v).strip().casefold()
+    if bare in placeholders or (needs_letter and not any(c.isalpha() for c in v)):
+        return ""
+    return v
+
+
+# The characters Plex documents as breaking playback. They are dropped, not
+# substituted: a replacement character is a guess at what the title meant.
+BAD_CHARS = set('<>:"/\\|?*')
+TRASH_DIR = re.compile(r"(^|/)(\.AGENTTRASH|\.thumbnails|\.Trash-\d+|@eaDir"
+                       r"|lost\+found)(/|$)")
+
+ARGS = None
+HELD = {}                            # path -> (bytes, ext)
+ROWS = {}                            # path -> the chapter row Plex matched
+PLACED = {}                          # chapter path -> the destination the plan gave it
+CLAIMED = set()                      # every destination path the plan has handed out
+MOVES, KEEPS = [], []
+PARKED = defaultdict(list)           # reason -> [path]
+NOTES = defaultdict(list)            # section -> [line]
+AUDIO_BY_DIR = defaultdict(set)      # source dir -> chapter files a row names
+CHILD_OF = defaultdict(set)          # dir -> the dirs directly below it holding chapters
+FOLDERS_SEEN = {}                    # (dir, depth) -> book folders placed below it
+SUBTREE_DEPTH = 2                    # how far below a folder its chapters may sit
+BOOKS = {}                           # (narrator, book, year) -> the book group
+
+AMBIGUOUS = object()
+
+
+def dominant(values):
+    """The spelling the rows mostly agree on: the most frequent value, ties broken
+    by sort so the answer never depends on the order rows arrived in."""
+    counts = Counter(values)
+    return min(counts.items(), key=lambda kv: (-kv[1], kv[0]))[0]
+
+
+def rel(path):
+    """The path as seen from the library root."""
+    return os.path.relpath(path, ARGS.root)
+
+
+def safe(name):
+    """A name every client can carry: no reserved character, nothing unprintable,
+    no trailing dot or space, and short enough to rename."""
+    out = "".join(c for c in (name or "")
+                  if c not in BAD_CHARS and (c.isprintable() or c == " "))
+    out = re.sub(r"\s+", " ", out).strip().rstrip(". ")
+    while out and len(out.encode("utf-8", "ignore")) > 200:
+        out = out[:-4].rstrip()
+    return out
+
+
+def assign(src, folder, name, ext, kind, note):
+    """Book one file into `folder` under `name`, renamed if that exact destination
+    is already taken — by another file in the plan, or by a file standing there
+    today. Two files claiming one path is the fault that costs a re-run of a 60k
+    plan, and so is a plan that only works if the renames happen in the right
+    order: the executor refuses any target that is occupied, so an occupied path
+    is never a destination, even when its occupant is scheduled to leave.
+    A file already standing where the contract puts it is a keep, not a move."""
+    stem = safe(name) or "file"
+    dst = os.path.join(folder, stem + ext)
+    if dst == src:
+        CLAIMED.add(dst.lower())          # it is standing on its own target
+        KEEPS.append((src, dst, kind))
+        return dst
+    # A file already in this folder under this name — or under the numbered
+    # variant the collision rule gives it — keeps the name it has. Renumbering it
+    # every run would turn the plan into a shuffle instead of a fixed point.
+    base = os.path.splitext(os.path.basename(src))[0]
+    if os.path.dirname(src) == folder and (base == stem
+                                           or base.startswith(stem + " (")):
+        CLAIMED.add(src.lower())
+        KEEPS.append((src, src, kind))
+        return src
+    n = 1
+    while dst.lower() in CLAIMED:
+        n += 1
+        dst = os.path.join(folder, f"{stem} ({n}){ext}")
+    if n > 1:
+        NOTES[note].append(rel(dst))
+    CLAIMED.add(dst.lower())
+    MOVES.append((src, dst, kind))
+    return dst
+
+
+def park(path, reason):
+    """Hold a file where it is, under the park folder, keeping its relative path."""
+    PARKED[reason].append(path)
+    dst = os.path.join(ARGS.park_dir, rel(path))
+    if dst != path:
+        CLAIMED.add(dst.lower())
+        MOVES.append((path, dst, "park"))
+
+
+def under_park(path):
+    """True for a file already parked: that pile is for a human to review, and
+    re-parking it every run would only push it deeper."""
+    return path.startswith(ARGS.park_dir + os.sep)
+
+
+# ------------------------------------------------------------------------ books
+def group_books():
+    """Every matched chapter into one group per (narrator, book, year).
+
+    The key is the identity Plex asserts in its own database, not the folder the
+    file sits in: one book arrives as four part folders inside a collection pack,
+    and two copies of one book in one pack are one book to Plex, whose album item
+    IS that narrator, title and year. The year is part of the key so a 1965
+    recording and a 2020 recording that share a title stay two books.
+    """
+    for path, r in sorted(ROWS.items()):
+        if path not in HELD or under_park(path) or TRASH_DIR.search(path):
+            continue
+        d = os.path.dirname(path)
+        AUDIO_BY_DIR[d].add(path)
+        CHILD_OF[os.path.dirname(d)].add(d)   # so a cover above Part 1/Part 2 sees both
+        narrator = identity(r["narrator"], PLACEHOLDER_NARRATOR, needs_letter=True)
+        book = identity(r["book"], PLACEHOLDER_BOOK)
+        if not book:
+            park(path, "a chapter row that names no book — Plex's album title is a "
+                       "placeholder like [Unknown Album], so filing on it would "
+                       "merge unrelated books onto one shelf")
+            continue
+        if not narrator:
+            park(path, "a chapter row that names no narrator — the artist row is a "
+                       "placeholder like [Unknown Artist] or a chapter number a "
+                       "ripper mistagged as the artist, so no shelf owns it")
+            continue
+        key = (narrator.casefold(), book.casefold(), (r["year"] or "").strip())
+        g = BOOKS.setdefault(key, {"narrators": [], "books": [], "year": key[2],
+                                   "members": [], "indexed": 0})
+        g["members"].append(path)
+        g["narrators"].append(narrator)
+        g["books"].append(book)
+        if str(r["track"] or "").isdigit():
+            g["indexed"] += 1
+
+
+def book_folder(g):
+    """The shelf Plex's own rows name: <narrator>/<book> (<year>). None when a
+    name is made of characters no path can hold — inventing a shelf is filing it
+    wrong, so those chapters park instead."""
+    narrator = safe(dominant(g["narrators"]))
+    book = safe(dominant(g["books"]))
+    if not narrator or not book:
+        return None
+    return os.path.join(ARGS.root, narrator,
+                        f"{book} ({g['year']})" if g["year"] else book)
+
+
+def place_books():
+    """One folder per book; every chapter keeps the name it has today."""
+    for key in sorted(BOOKS):
+        g = BOOKS[key]
+        folder = book_folder(g)
+        if folder is None:
+            for path in sorted(g["members"]):
+                park(path, "a book or narrator Plex spells with characters no "
+                           "folder name can hold")
+            continue
+        for path in sorted(g["members"]):
+            ext = HELD[path][1]
+            base = os.path.splitext(os.path.basename(path))[0]
+            # The name the ripper wrote, unchanged: the track tag inside the file
+            # is what orders playback, and 72% of the books here have chapters no
+            # row numbers, so any number added here would be invented.
+            PLACED[path] = assign(path, folder, base, ext, "chapter",
+                                  "two chapters of one book carried the same name")
+
+
+# --------------------------------------------------------------------- sidecars
+def folders_below(d, depth):
+    """The book folders the chapters in `d`, or within `depth` directories below
+    it, were placed in. Memoised, because every loose file in a big collection
+    folder asks the same question about it."""
+    if (d, depth) not in FOLDERS_SEEN:
+        seen = {os.path.dirname(PLACED[a]) for a in AUDIO_BY_DIR.get(d, ())
+                if a in PLACED}
+        if depth:
+            for child in CHILD_OF.get(d, ()):
+                seen |= folders_below(child, depth - 1)
+        FOLDERS_SEEN[(d, depth)] = seen
+    return FOLDERS_SEEN[(d, depth)]
+
+
+def owner_folder(d):
+    """The book folder that owns this directory — the first place up the tree
+    where chapters sit, in the directory itself or within SUBTREE_DEPTH levels
+    below it, which is how a pack whose parts are `… [Part 1]`/`… [Part 2]` owns
+    the cover art lying above them. None when no chapter is near enough to say,
+    AMBIGUOUS when the chapters there belong to more than one book."""
+    while d.startswith(ARGS.root + os.sep) and d != ARGS.root:
+        folders = folders_below(d, SUBTREE_DEPTH)
+        if folders:
+            return next(iter(folders)) if len(folders) == 1 else AMBIGUOUS
+        d = os.path.dirname(d)
+    return None
+
+
+def place_other():
+    """Everything that is not a chapter Plex matched: covers, booklets, EPUBs,
+    logs, checksums, playlists — and the clutter that is none of those."""
+    for path, (_size, ext) in sorted(HELD.items()):
+        if path in ROWS or under_park(path) or TRASH_DIR.search(path):
+            continue
+        d = os.path.dirname(path)
+        base = os.path.splitext(os.path.basename(path))[0]
+        if ext in VIDEO_EXT:
+            park(path, "a video container inside the audiobook library: Plex reads "
+                       "it as video, and renaming it would claim it is a chapter")
+        elif ext in AUDIO_EXT:
+            park(path, "an audio file no chapter row names — it is not in the "
+                       "library Plex matched, so nothing says which book it is")
+        else:
+            owner = owner_folder(d)
+            if owner is AMBIGUOUS:
+                park(path, "a file whose folder holds more than one book, so "
+                           "nothing says which one it belongs to")
+            elif owner:
+                assign(path, owner, base, ext,
+                       "playlist" if ext in (".m3u", ".m3u8") else "sidecar",
+                       "one book folder would have carried two files of the same"
+                       " sidecar name")
+            else:
+                park(path, "not an audiobook file, and no book's folder lies with"
+                           " it to carry it")
+
+
+# ---------------------------------------------------------------------- outputs
+def write_plan():
+    """mkdir rows for every destination folder and its whole chain, then the moves.
+    Parents first: the executor creates directories in the order they are listed."""
+    chain = set()
+    for _s, dst, _k in MOVES:
+        d = os.path.dirname(dst)
+        while d and d != ARGS.root and d != os.path.dirname(d):
+            chain.add(d)
+            d = os.path.dirname(d)
+    dirs = sorted(chain, key=lambda p: (p.count("/"), p))
+    with open(ARGS.out, "w") as fh:
+        fh.write("action\tsrc\tdst\tkind\n")
+        for d in dirs:
+            fh.write(f"mkdir\t\t{d}\tdir\n")
+        for src, dst, kind in sorted(MOVES, key=lambda m: (m[1], m[0])):
+            fh.write(f"move\t{src}\t{dst}\t{kind}\n")
+        for src, dst, kind in sorted(KEEPS, key=lambda m: (m[1], m[0])):
+            fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
+    parked = [m for m in MOVES if m[2] == "park"]
+    sized = sum(HELD[s][0] for s, _d, _k in MOVES if s in HELD)
+    print(f"plan: {len(MOVES) - len(parked)} moves, {len(parked)} parked"
+          f" ({sized / 2**40:.2f} TiB touched), {len(KEEPS)} already placed,"
+          f" {len(dirs)} folders -> {ARGS.out}")
+
+
+def write_exceptions():
+    with open(ARGS.exceptions, "w") as fh:
+        fh.write(f"Audiobook relayout exceptions for {ARGS.root}\n\n")
+        fh.write(f"Everything below is parked under {ARGS.park_dir}, keeping the\n"
+                 "relative path it has today. Nothing is deleted and nothing is\n"
+                 "overwritten: a parked file is one rename from wherever the plan is\n"
+                 "pointed next.\n\n")
+        for reason in sorted(PARKED, key=lambda r: -len(PARKED[r])):
+            rs = PARKED[reason]
+            fh.write(f"{len(rs):>6} files  {reason}\n")
+            for p in sorted(rs, key=rel)[:400]:
+                fh.write(f"          {rel(p)}\n")
+            if len(rs) > 400:
+                fh.write(f"          … and {len(rs) - 400} more\n")
+            fh.write("\n")
+        for section in sorted(NOTES, key=lambda s: -len(NOTES[s])):
+            lines = sorted(set(NOTES[section]))
+            fh.write(f"{len(lines):>6}  {section}\n")
+            for line in lines[:200]:
+                fh.write(f"          {line}\n")
+            if len(lines) > 200:
+                fh.write(f"          … and {len(lines) - 200} more\n")
+            fh.write("\n")
+        unindexed = sorted(f"{safe(dominant(g['narrators']))}/{safe(dominant(g['books']))}"
+                           for g in BOOKS.values()
+                           if 0 < g["indexed"] < len(g["members"]))
+        if unindexed:
+            fh.write(f"{len(unindexed):>6} books have chapters Plex does not number. Their\n")
+            fh.write("       files keep the names they carry: playback order comes from the\n"
+                     "       track tag inside each file, and a number written here would be\n"
+                     "       invented for the chapters with no index. Fix it in the tags, not\n"
+                     "       in the names — Plex re-reads tags on a scan and never names files.\n\n")
+        fh.write("       Playlists move with their book and their contents are never\n"
+                 "       rewritten: the paths inside an .m3u go stale.\n")
+    print(f"exceptions -> {ARGS.exceptions}")
+
+
+def main():
+    global ARGS
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default=os.environ.get("MEDIA_AUDIOBOOKS_ROOT",
+                                                     "/mnt/entropy/Audiobooks"))
+    ap.add_argument("--enum", default="/tmp/audiobooks-enum.tsv",
+                    help="the enumerate TSV (bytes, ext, path)")
+    ap.add_argument("--rows", default="/tmp/plex-audiobooks.tsv",
+                    help="the chapter rows scripts/media-census.py audiobooks wrote")
+    ap.add_argument("--out", default="/tmp/audiobooks-plan.tsv")
+    ap.add_argument("--exceptions", default="/tmp/audiobooks-exceptions.txt")
+    ap.add_argument("--park", default="_unsorted",
+                    help="the folder inside the root that holds what cannot be filed")
+    ARGS = ap.parse_args()
+    ARGS.root = os.path.normpath(ARGS.root)
+    ARGS.park_dir = os.path.join(ARGS.root, ARGS.park)
+    read_inputs()
+    # Every path a file occupies today is claimed before a single destination is
+    # handed out: the executor refuses an occupied target, so a plan that aimed at
+    # one would only fail after the renames that were supposed to clear it.
+    CLAIMED.update(p.lower() for p in HELD)
+    group_books()
+    place_books()
+    place_other()
+    write_plan()
+    write_exceptions()
+
+
+def read_inputs():
+    with open(ARGS.enum) as fh:
+        for line in fh:
+            parts = line.rstrip("\n").split("\t")
+            if len(parts) >= 3:
+                HELD[parts[2]] = (int(parts[0]), parts[1].lower())
+    with open(ARGS.rows, newline="") as fh:
+        for row in csv.DictReader(fh, delimiter="\t"):
+            if row.get("file"):
+                ROWS[row["file"]] = row
+    gone = [p for p in ROWS if p not in HELD]
+    if gone:
+        NOTES["a chapter row whose file the census no longer sees"].extend(gone)
+
+
+if __name__ == "__main__":
+    main()

scripts/media-census.py

@@ -1,13 +1,15 @@
 #!/usr/bin/env python3
-r"""Read-only census of a movie library — the inputs a relayout plan is built from.
+r"""Read-only census of a media library — the inputs a relayout plan is built from.
 
   media-census.py enumerate [--root DIR] [--out TSV]
   media-census.py probe     [--enum TSV] [--out TSV] [--jobs N]
   media-census.py plex      [--root DIR] [--db PATH] [--out TSV]
+  media-census.py jellyfin  [--root DIR] [--db PATH] [--container PREFIX] [--out TSV]
+  media-census.py audiobooks [--root DIR] [--db PATH] [--out TSV]
 
 Run it on the machine that mounts the library. Nothing here writes inside the
-library and nothing is deleted; the only outputs are the three TSVs below, and
-scripts/media-plan.py reads them instead of touching media.
+library and nothing is deleted; the only outputs are the TSVs below, and the
+scripts/media-plan.py and scripts/media-shows.py read them instead of touching media.
 
   enum    bytes <TAB> ext <TAB> path                  every file under the root
   probed  bytes <TAB> path <TAB> dur <TAB> vcodec <TAB> WxH <TAB> fps <TAB> vbit
@@ -18,10 +20,34 @@ scripts/media-plan.py reads them instead of touching media.
   plex    file <TAB> leaf_title <TAB> … (see PLEX_COLS)
           the titles Plex already matched — the authority over any filename guess
 
+  jellyfin file <TAB> series <TAB> year <TAB> season <TAB> episode <TAB> extra
+          <TAB> tmdb <TAB> title <TAB> tvdb      (see JF_COLS)
+          one row per episode Jellyfin has matched and numbered, its path mapped
+          from the container's view back to the host; `year`, `tmdb` and `tvdb`
+          are the SERIES' values, `title` is the episode title the row carries,
+          `extra` is Jellyfin's own extra type when the row is bonus material
+          rather than an episode
+
+  music file <TAB> artist <TAB> album <TAB> year <TAB> disc <TAB> track <TAB> title
+         <TAB> mb <TAB> performer   (see MUSIC_COLS)
+         one row per track Jellyfin has tagged: the tag read is what the row IS,
+         so `artist` is the album artist (the track artist when that is empty) and
+         `album`/`year` are the album item's provider values when the folder is an
+         album item and the tags' otherwise; `mb` is the album's MusicBrainz
+         release group, `performer` the track artist the tags carry
+
+  audiobooks file <TAB> narrator <TAB> book <TAB> year <TAB> track <TAB> chapter
+         (see AB_COLS)
+         one row per chapter Plex matched under the audiobook root: `book` is the
+         album row Plex built from the files' own tags, `narrator` that book's
+         artist row — Plex keeps exactly one artist per book, which is why it, and
+         not an author parsed out of a title, names the shelf. `track` is empty
+         wherever Plex has no chapter number, which is most of this library.
+
 `probe` is the slow step (every container is opened for its header, ~10 min for
 4,800 files on a spindle array at 6 workers) so it is `nice`d: a census beside a
 live service must not starve it. `enumerate` is a stat pass, so re-censusing
-before a plan is cheap — and a plan is only ever as good as these three files.
+before a plan is cheap — and a plan is only ever as good as these files.
 """
 import argparse
 import csv
@@ -57,6 +83,72 @@ join metadata_items mi on mi.id = m.metadata_item_id
 left join metadata_items par on par.id = mi.parent_id
 where p.file like ? and p.deleted_at is null and mi.deleted_at is null
 """
+JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
+JF_COLS = ["file", "series", "year", "season", "episode", "extra",
+           "tmdb", "title", "tvdb"]
+# Jellyfin stores the entity's .NET class as its type; the TV tree is these three.
+JF_TV = ("MediaBrowser.Controller.Entities.TV.Episode",
+         "MediaBrowser.Controller.Entities.TV.Season",
+         "MediaBrowser.Controller.Entities.TV.Series")
+JF_SQL = ("""select Id, Path, IndexNumber, ParentIndexNumber, ExtraType,
+                   IsVirtualItem, Name from BaseItems where Type = ? and Path like ?""")
+
+# The music tree is these two entity classes. The track row carries the tags
+# Jellyfin read out of the file; the album row carries what the provider matched,
+# which is the better folder name of the two.
+JF_AUDIO = "MediaBrowser.Controller.Entities.Audio.Audio"
+JF_ALBUM = "MediaBrowser.Controller.Entities.Audio.MusicAlbum"
+MUSIC_COLS = ["file", "artist", "album", "year", "disc", "track", "title", "mb",
+              "performer"]
+MUSIC_SQL = ("""select b.Path, b.AlbumArtists, b.Artists, b.Album, b.ProductionYear,
+                       b.ParentIndexNumber, b.IndexNumber, b.Name, b.Id,
+                       a.Name, a.ProductionYear, a.Id
+                  from BaseItems b
+                  left join BaseItems a on a.Id = b.ParentId and a.Type = ?
+                 where b.Type = ? and b.Path like ?""")
+# The album's identity, in preference order. A release group is the album; a bare
+# release id is one reissue of it.
+MUSIC_PROVIDERS = ("MusicBrainzReleaseGroup", "MusicBrainzAlbum")
+
+# Plex's audiobook tree is the same Artist/Album/Track shape as its music tree:
+# the leaf (10) is a CHAPTER, its parent (9) is the BOOK, and that book's artist
+# (8) is the NARRATOR. The track's own `index` is exported because the exceptions
+# report counts how many books have chapters Plex does not number — the reason the
+# relayout refuses to invent numbers into file names.
+PLEX_ARTIST, PLEX_ALBUM, PLEX_TRACK = 8, 9, 10
+AB_COLS = ["file", "narrator", "book", "year", "track", "chapter"]
+AB_SQL = f"""
+select p.file, coalesce(ar.title,''), coalesce(al.title,''), coalesce(al.year,0),
+       coalesce(mi."index",-1), coalesce(mi.title,'')
+from media_parts p
+join media_items m on m.id = p.media_item_id
+join metadata_items mi on mi.id = m.metadata_item_id
+join metadata_items al on al.id = mi.parent_id and al.metadata_type = {PLEX_ALBUM}
+left join metadata_items ar on ar.id = al.parent_id
+                          and ar.metadata_type = {PLEX_ARTIST}
+where p.file like ? and p.deleted_at is null and mi.deleted_at is null
+  and al.deleted_at is null
+"""
+
+def open_plex_ro(db):
+    """Plex's library, read the way Plex writes it.
+
+    `immutable=1` tells sqlite the file never changes, so it skips the
+    write-ahead log: every row a scan has committed but not yet checkpointed
+    is invisible, and reading a live database that way can fail outright as
+    "database disk image is malformed". Plain ro reads the WAL through its
+    -shm; immutable is only the fallback for a database whose -shm cannot be
+    read. The same law scripts/media-verify.py states.
+    """
+    last = None
+    for opt in ("mode=ro", "mode=ro&immutable=1"):
+        try:
+            return sqlite3.connect(f"file:{db}?{opt}", uri=True)
+        except sqlite3.Error as e:
+            last = e
+    sys.exit(f"cannot open the Plex database read-only ({db}): {last}\n"
+             "Plex runs rootless here: run this as the container's owner, never "
+             "with sudo — rootless storage is per-user, so root sees nothing.")
 
 
 def enumerate_lib(root, out):
@@ -157,15 +249,9 @@ def probe(enum, out, jobs):
 
 def plex(root, db, out):
     """Plex's matched titles for files under root, from its DB, read-only."""
-    uri = f"file:{db}?mode=ro&immutable=1"
-    try:
-        con = sqlite3.connect(uri, uri=True)
-    except sqlite3.Error as e:
-        sys.exit(f"cannot open the Plex database read-only ({db}): {e}\n"
-                 "Plex runs rootless here: run this as the container's owner, never "
-                 "with sudo — rootless storage is per-user, so root sees nothing.")
+    con = open_plex_ro(db)
     with open(out, "w", newline="") as fh:
-        w = csv.writer(fh, delimiter="\t")
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
         w.writerow(PLEX_COLS)
         n = 0
         for r in con.execute(PLEX_SQL, (root + "/%",)):
@@ -174,6 +260,210 @@ def plex(root, db, out):
     print(f"plex-matched={n} -> {out}")
 
 
+def jellyfin(root, db, container, out):
+    """The episodes Jellyfin has already numbered, for the library at root.
+
+    Jellyfin stores each path as the CONTAINER sees it (its quadlet binds the
+    share at /data), so `container` is the in-container prefix that stands for
+    root and every exported path is mapped back to the host before it is written
+    — the planner reads host paths, because it plans moves on the array.
+
+    The numbering is what matters, not the name: a release group's file name says
+    nothing about which episode a file is, and Jellyfin has already answered that
+    question against a provider. Season/episode and the episode's own title come
+    from the episode row; the series name, its year and its TMDb and Tvdb ids come
+    from the Series row the episode climbs to, because those are the only place
+    Jellyfin keeps them.
+    """
+    try:
+        con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+    except sqlite3.Error as e:
+        sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
+                 "It is a WAL database: copy it with its -wal and -shm beside it and"
+                 " point --db at the copy. Never sudo it — the data dir belongs to"
+                 " the container's owner, and rootless storage is per-user.")
+    have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
+    need = {"Id", "ParentId", "Type", "Path", "IndexNumber", "ParentIndexNumber",
+            "ExtraType", "IsVirtualItem", "ProductionYear", "Data"}
+    if not need <= have:
+        sys.exit(f"{db} is not a Jellyfin library database (BaseItems has no"
+                 f" {','.join(sorted(need - have))})")
+    node = {}
+    for i, pid, typ, name, year, data in con.execute(
+            "select Id, ParentId, Type, Name, ProductionYear, Data from BaseItems"
+            " where Type in (?,?,?)", JF_TV):
+        node[i] = (pid, typ, name, year, data)
+
+    # Jellyfin 12 keeps provider ids in their own table; older builds kept them in
+    # the row's Data blob. Both shapes are read, because the tvdb id the series
+    # naming depends on is wherever this build happened to write it.
+    prov = {}
+    if "BaseItemProviders" in {r[0] for r in con.execute(
+            "select name from sqlite_master where type='table'")}:
+        for iid, pid, val in con.execute(
+                "select ItemId, ProviderId, ProviderValue from BaseItemProviders"
+                " where ProviderId in ('Tmdb', 'Tvdb')"):
+            prov[(iid, pid)] = val
+
+    def provider_of(item_id, want):
+        got = prov.get((item_id, want))
+        if got:
+            return str(got)
+        try:
+            blob = json.loads(node[item_id][4] or "{}").get("ProviderIds") or {}
+        except (KeyError, ValueError, TypeError):
+            return ""
+        return str(blob.get(want, "") or "")
+
+    def series_of(item_id):
+        """Climb episode -> season -> series; empty when the chain is broken."""
+        for _ in range(4):
+            if item_id not in node:
+                return "", "", "", ""
+            pid, typ = node[item_id][0], node[item_id][1]
+            if typ.endswith(".Series"):
+                return (node[item_id][2] or "", node[item_id][3] or "",
+                        provider_of(item_id, "Tmdb"), provider_of(item_id, "Tvdb"))
+            item_id = pid
+        return "", "", "", ""
+
+    # The fleet's quadlets bind the share at /data, so the container's view of
+    # this library is /data/<the library's own folder name> unless told otherwise.
+    if not container:
+        container = "/data/" + os.path.basename(os.path.normpath(root))
+    prefix = container.rstrip("/") + "/"
+    written = skipped = 0
+    with open(out, "w", newline="") as fh:
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
+        w.writerow(JF_COLS)
+        for iid, path, idx, pidx, extra, virt, epname in con.execute(
+                JF_SQL, (JF_TV[0], prefix + "%")):
+            if virt or not path or not path.startswith(prefix):
+                skipped += 1          # a virtual item is a variant, not a file
+                continue
+            sname, syear, tmdb, tvdb = series_of(iid)
+            w.writerow([root + "/" + path[len(prefix):],
+                        sname.replace("\t", " "), syear,
+                        "" if pidx is None else pidx,
+                        "" if idx is None else idx,
+                        (extra or "").replace("\t", " "), tmdb,
+                        (epname or "").replace("\t", " "), tvdb])
+            written += 1
+    print(f"episodes={written} virtual-or-blank={skipped} -> {out}")
+
+
+def music(root, db, container, out):
+    """The tracks Jellyfin has already tagged, for the library at root.
+
+    Jellyfin reads music metadata out of the file itself, so an Audio row IS the
+    tag read, done once, by the vendor's own parser: its album-artist, album,
+    year, disc and track columns are what the embedded tags say and its Name is
+    the track title. That makes this export the whole authority for a music
+    relayout — the path a ripper chose is not evidence of anything, and the
+    folder Jellyfin put a track under is often a release pack, not an album.
+
+    Two gaps this export answers from the row rather than the tree. Many tracks
+    sit under a folder that is not an album item at all (a disc folder, or a
+    mixed one), so the album name and year fall back to the tags when the album
+    row is absent. And an album's MusicBrainz ids are not in the row's own Data
+    — the current schema leaves that empty for music — they are rows of
+    BaseItemProviders keyed by the item's Id.
+    """
+    try:
+        con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+    except sqlite3.Error as e:
+        sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
+                 "It is a WAL database: copy it with its -wal and -shm beside it and"
+                 " point --db at the copy. Never sudo it — the data dir belongs to"
+                 " the container's owner, and rootless storage is per-user.")
+    have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
+    need = {"Id", "ParentId", "Type", "Path", "Name", "IndexNumber",
+            "ParentIndexNumber", "ProductionYear", "Album", "AlbumArtists",
+            "Artists"}
+    if not need <= have:
+        sys.exit(f"{db} is not a Jellyfin library database with music columns"
+                 f" (BaseItems has no {','.join(sorted(need - have))})")
+    # Provider ids, best identity first: the release group is the album, a bare
+    # release id is one reissue of it and never overrides a group.
+    mb = {}
+    if "BaseItemProviders" in {r[0] for r in con.execute(
+            "select name from sqlite_master where type='table'")}:
+        rank = {p: i for i, p in enumerate(MUSIC_PROVIDERS)}
+        best = {}
+        for iid, pid, val in con.execute(
+                "select ItemId, ProviderId, ProviderValue from BaseItemProviders"
+                " where ProviderId in (?, ?)", MUSIC_PROVIDERS):
+            if iid not in best or rank[pid] < best[iid][0]:
+                best[iid] = (rank[pid], val)
+        mb = {k: v[1] for k, v in best.items()}
+
+    # The fleet's quadlets bind the share at /data, so the container's view of
+    # this library is /data/<the library's own folder name> unless told otherwise.
+    if not container:
+        container = "/data/" + os.path.basename(os.path.normpath(root))
+    prefix = container.rstrip("/") + "/"
+    written = skipped = 0
+    with open(out, "w", newline="") as fh:
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
+        w.writerow(MUSIC_COLS)
+        for (path, aal, arts, tag_album, tag_year, disc, track, name, iid,
+             alb_name, alb_year, alb_id) in con.execute(
+                MUSIC_SQL, (JF_ALBUM, JF_AUDIO, prefix + "%")):
+            if not path or not path.startswith(prefix):
+                skipped += 1          # a row for a file outside this library
+                continue
+            # The album artist owns the folder; the track artist is the fallback
+            # for the compilations whose album-artist tag was left empty.
+            w.writerow([root + "/" + path[len(prefix):],
+                        ((aal or "").strip() or (arts or "").strip()),
+                        (alb_name or tag_album or "").replace("\t", " "),
+                        alb_year or tag_year or "",
+                        "" if disc is None else disc,
+                        "" if track is None else track,
+                        (name or "").replace("\t", " "),
+                        mb.get(alb_id or "") or mb.get(iid or "") or "",
+                        (arts or "").replace("\t", " ")])
+            written += 1
+    print(f"tracks={written} outside-root-or-blank={skipped} -> {out}")
+
+
+def audiobooks(root, db, out):
+    """The chapters Plex matched inside the audiobook library, read-only.
+
+    Plex stores HOST paths here (unlike Jellyfin, which stores the container's
+    view), so no prefix mapping is needed: the part's `file` is the path the
+    operator has today. Rows for files outside `root` are skipped, because one
+    Plex library DB carries every library on the box.
+
+    The chapter's `index` is exported as an empty field when Plex has none. That
+    is not a gap to fill in here: three quarters of the books on this box contain
+    at least one un-indexed chapter, so the relayout that reads this file must
+    not renumber anything from it.
+    """
+    con = open_plex_ro(db)
+    tables = {r[0] for r in con.execute(
+        "select name from sqlite_master where type='table'")}
+    missing = {"media_parts", "media_items", "metadata_items"} - tables
+    if missing:
+        sys.exit(f"{db} is not a Plex library database (no "
+                 f"{', '.join(sorted(missing))})")
+    written = skipped = 0
+    with open(out, "w", newline="") as fh:
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
+        w.writerow(AB_COLS)
+        for file, narrator, book, year, track, chapter in con.execute(
+                AB_SQL, (root.rstrip("/") + "/%",)):
+            if not file or not file.startswith(root.rstrip("/") + "/"):
+                skipped += 1              # a row for a file outside this library
+                continue
+            w.writerow([file.replace("\t", " "), narrator.replace("\t", " "),
+                        book.replace("\t", " "), "" if not year else year,
+                        "" if track is None or int(track) < 0 else track,
+                        chapter.replace("\t", " ")])
+            written += 1
+    print(f"chapters={written} outside-root={skipped} -> {out}")
+
+
 def main():
     ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
     sub = ap.add_subparsers(dest="cmd", required=True)
@@ -190,13 +480,46 @@ def main():
     x.add_argument("--root", **root)
     x.add_argument("--db", default=PLEX_DB)
     x.add_argument("--out", default="/tmp/plex-movies.tsv")
+    j = sub.add_parser("jellyfin",
+                       help="export the episodes Jellyfin has numbered for the root")
+    j.add_argument("--root", default=os.environ.get("MEDIA_ROOT",
+                                                    "/mnt/entropy/TVShows"),
+                   help="TV library root on the host")
+    j.add_argument("--db", default=JF_DB)
+    j.add_argument("--container", default=None,
+                   help="the in-container prefix that stands for --root"
+                        " (default /data/<root's folder name>)")
+    j.add_argument("--out", default="/tmp/jellyfin-tv.tsv")
+    m = sub.add_parser("music",
+                       help="export the tracks Jellyfin has tagged for the root")
+    m.add_argument("--root", default=os.environ.get("MEDIA_MUSIC_ROOT",
+                                                    "/mnt/entropy/Music"),
+                   help="music library root on the host")
+    m.add_argument("--db", default=JF_DB)
+    m.add_argument("--container", default=None,
+                   help="the in-container prefix that stands for --root"
+                        " (default /data/<root's folder name>)")
+    m.add_argument("--out", default="/tmp/jellyfin-music.tsv")
+    b = sub.add_parser("audiobooks",
+                       help="export the chapters Plex matched for the audiobook root")
+    b.add_argument("--root", default=os.environ.get("MEDIA_AUDIOBOOKS_ROOT",
+                                                    "/mnt/entropy/Audiobooks"),
+                   help="audiobook library root on the host")
+    b.add_argument("--db", default=PLEX_DB)
+    b.add_argument("--out", default="/tmp/plex-audiobooks.tsv")
     a = ap.parse_args()
     if a.cmd == "enumerate":
         enumerate_lib(a.root, a.out)
     elif a.cmd == "probe":
         probe(a.enum, a.out, a.jobs)
-    else:
+    elif a.cmd == "plex":
         plex(a.root, a.db, a.out)
+    elif a.cmd == "music":
+        music(a.root, a.db, a.container, a.out)
+    elif a.cmd == "audiobooks":
+        audiobooks(a.root, a.db, a.out)
+    else:
+        jellyfin(a.root, a.db, a.container, a.out)
 
 
 if __name__ == "__main__":

scripts/media-ebook-dedupe.py

@@ -0,0 +1,239 @@
+#!/usr/bin/env python3
+"""media-ebook-dedupe.py — dedupe the eBooks pile against the Calibre library.
+
+2026-11-04 campaign, ebook side. The Calibre library (/mnt/entropy/CalibreBooks,
+6,289 books) is DB-owned and healthy; /mnt/entropy/eBooks is a raw pile whose
+best books already live in the library, several of them twice in the pile.
+Audiobooks' orphaned ebooks/comics land in the pile first (media-ab-reorg's
+intake), so this campaign runs after the audiobook move.
+
+Inputs
+  --meta      ebook-meta.tsv: JSON lines {p, sz, title, authors, ident} for
+              epub (embedded OPF) and mobi/azw3 (Palm header title); the
+              producer is the campaign's opfcensus.py.
+  --enum      enum TSV (size, ext, path) for every tree.
+  --calibre   calibre-books.tsv: JSON lines [id, title, authors, series, path,
+              pubdate, formats] dumped from metadata.db.
+  --dups      full blake2b verdicts (hash, size, path) — only confirmed
+              byte-identities remove a file. Optional: without it, identity
+              rules alone run and every hash-dependent pair stays a KEEP.
+
+Outputs
+  --plan      media-apply plan (--root /mnt/entropy): hash-duplicates of a
+              file already in CalibreBooks -> HOLD; byte-identical files inside
+              the pile keep the best format of one book once; non-book junk
+              (mp4/mp3/images) -> PARK.
+  --ingest    one JSON line per surviving BOOK (a set of format files of one
+              identity): {paths, title, authors, source}. The ingest runner
+              feeds these to calibredb inside the calibre-web image.
+  --exceptions every identity pair the file-level evidence could not settle.
+
+Match law: identity is (core title, first author surname). Calibre's
+"Dinniman| Matt" and an OPF "Matt Dinniman" share surname "dinniman". Title
+core drops the leading article and all non-alphanumerics. Identity alone NEVER
+deletes against the pile — it only deletes when the same identity sits in the
+Calibre library with the same format (the pile copy adds nothing), or when a
+byte-hash confirms the pair. Format preference when the pile holds one book in
+several copies: epub > azw3 > mobi > pdf > fb2 > lit > rtf > txt.
+"""
+import argparse
+import json
+import os
+import re
+import unicodedata
+from collections import defaultdict
+
+BOOK_EXT = {"epub", "azw3", "mobi", "pdf", "fb2", "lit", "rtf", "txt", "cbr", "cbz"}
+FORMAT_RANK = {"epub": 0, "azw3": 1, "mobi": 2, "pdf": 3, "fb2": 4,
+               "cbr": 5, "cbz": 5, "lit": 6, "rtf": 7, "txt": 8}
+ARTICLE = re.compile(r"^(the|a|an)\s+", re.I)
+
+
+def ascii_fold(s):
+    return "".join(c for c in unicodedata.normalize("NFKD", s or "")
+                   if not unicodedata.combining(c))
+
+
+def title_core(title):
+    t = ascii_fold(title).lower()
+    t = ARTICLE.sub("", t)
+    t = re.sub(r"\(.*?\)", " ", t)          # (1998), (Unabridged) — noise
+    t = re.sub(r"[^a-z0-9 ]", " ", t)
+    t = re.sub(r"\b(1[5-9]|20)\d\d\b", " ", t)  # years
+    return " ".join(t.split())[:120]
+
+
+def surname(author):
+    a = ascii_fold(author).strip()
+    a = re.split(r"[;,]", a)[0] if "," in a else " ".join(a.split()[-1:])
+    return re.sub(r"[^a-z]", "", a.lower())
+
+
+def first_author(authors):
+    parts = [p.strip() for p in re.split(r"[;&]", authors or "") if p.strip()]
+    return parts[0] if parts else ""
+
+
+def stem_title(path):
+    s = os.path.splitext(os.path.basename(path))[0]
+    return ascii_fold(s)
+
+
+class Campaign:
+    def __init__(self, args):
+        self.args = args
+        self.root = args.root.rstrip("/")
+        self.pile = self.root + "/eBooks"
+        self.lib = self.root + "/CalibreBooks"
+        self.hold = args.hold
+        self.park = args.park
+        self.meta = {}           # path -> meta dict
+        self.pile_files = []     # (sz, ext, path)
+        self.cal_by_path = {}    # file path under CalibreBooks -> calibre row
+        self.cal_identity = {}   # (tcore, surname) -> calibre row
+        self.drop = {}           # path -> reason
+        self.exc = defaultdict(list)
+        self.moves = []
+        self.dirs_pile_book = defaultdict(list)  # book dir (pile) -> files
+
+    # ------------------------------------------------------------- inputs
+    def load(self):
+        for line in open(self.args.meta):
+            r = json.loads(line)
+            self.meta[r["p"]] = r
+        by_hash = defaultdict(list)
+        if self.args.dups:
+            for line in open(self.args.dups):
+                h, sz, p = line.rstrip("\n").split("\t", 2)
+                by_hash[h].append(p)
+        self.dup_sets = [ps for ps in by_hash.values() if len(ps) > 1]
+        for line in open(self.args.enum):
+            sz, ext, p = line.rstrip("\n").split("\t", 2)
+            ext = ext.lstrip(".").lower()
+            if p.startswith(self.pile + "/") and p not in self.park:
+                self.pile_files.append((int(sz), ext, p))
+        for line in open(self.args.calibre):
+            row = json.loads(line)
+            self.cal_identity[(title_core(row[1]), surname(first_author(row[2])))] = row
+
+    def identity(self, p):
+        m = self.meta.get(p)
+        if m and m.get("title"):
+            t = title_core(m["title"])
+            a = surname(first_author(m.get("authors", "")))
+            if t:
+                return (t, a)
+        return (title_core(stem_title(p)), "")
+
+    # ------------------------------------------------------------- rules
+    def run(self):
+        pile_paths = {p for _, _, p in self.pile_files}
+        # R1/R2: confirmed byte-duplicates
+        for ps in self.dup_sets:
+            in_lib = [p for p in ps if p.startswith(self.lib + "/")]
+            in_pile = [p for p in ps if p.startswith(self.pile + "/")]
+            if in_lib:
+                for p in in_pile:
+                    self.drop[p] = "hash-duplicate of CalibreBooks file"
+            if len(in_pile) > 1:
+                keep = min(in_pile, key=lambda p: (FORMAT_RANK.get(
+                    os.path.splitext(p)[1].lstrip(".").lower(), 9), len(p), p))
+                for p in in_pile:
+                    if p != keep:
+                        self.drop[p] = "hash-duplicate inside the pile"
+        # R3/R4: identity against the library (same format only deletes)
+        cal_files = defaultdict(set)   # identity -> formats in the library
+        for key, row in self.cal_identity.items():
+            cal_files[key] = set(f for f in row[6].split("; ") if f)
+        for sz, ext, p in self.pile_files:
+            if p in self.drop or ext not in BOOK_EXT:
+                continue
+            key = self.identity(p)
+            if key[0] and key in self.cal_identity:
+                if ext in cal_files[key]:
+                    self.drop[p] = f"same identity+format already in Calibre ({key[0][:40]})"
+                else:
+                    self.exc["pile holds a format the library lacks (add-format on ingest)"].append(
+                        f"{p}\t{key[0][:50]}\t{self.cal_identity[key][0]}")
+        # R5: surviving files grouped by (identity, parent-ish book dir)
+        books = defaultdict(list)
+        for sz, ext, p in self.pile_files:
+            if p in self.drop:
+                continue
+            if ext not in BOOK_EXT:
+                self.moves.append((p, os.path.join(self.park, os.path.relpath(p, self.root)),
+                                   "non-book"))
+                continue
+            m = self.meta.get(p, {})
+            if m.get("err") and ext in ("epub",):
+                self.exc["unreadable container (park for a human)"].append(p)
+                self.moves.append((p, os.path.join(self.park, os.path.relpath(p, self.root)),
+                                   "broken-book"))
+                continue
+            key = self.identity(p)
+            books[key].append(p)
+        for key, ps in books.items():
+            first = min(ps, key=lambda p: (FORMAT_RANK.get(
+                os.path.splitext(p)[1].lstrip(".").lower(), 9), len(p), p))
+            m = self.meta.get(first, {})
+            self.ingest = {
+                "key": list(key),
+                "paths": sorted(ps),
+                "title": m.get("title") or stem_title(first),
+                "authors": m.get("authors") or "",
+                "ident": m.get("ident") or "",
+            }
+            self.ingest_list.append(self.ingest)
+        for p, why in self.drop.items():
+            self.moves.append((p, os.path.join(self.hold, os.path.relpath(p, self.root)), "dup"))
+
+    def write(self):
+        dirs = {os.path.dirname(d) for _, d, _ in self.moves}
+        all_dirs = set()
+        for d in dirs:
+            while d and d != self.root:
+                all_dirs.add(d)
+                d = os.path.dirname(d)
+        with open(self.args.plan, "w") as fh:
+            fh.write("action\tsrc\tdst\tkind\n")
+            for d in sorted(all_dirs):
+                fh.write(f"mkdir\t\t{d}\tcampaign\n")
+            for src, dst, kind in sorted(self.moves):
+                fh.write(f"move\t{src}\t{dst}\t{kind}\n")
+        with open(self.args.ingest, "w") as fh:
+            for b in self.ingest_list:
+                fh.write(json.dumps(b, ensure_ascii=False) + "\n")
+        with open(self.args.exceptions, "w") as fh:
+            for why in sorted(self.exc):
+                fh.write(f"# {why}\n")
+                for line in sorted(set(self.exc[why])):
+                    fh.write(line + "\n")
+                fh.write("\n")
+        ndrop = sum(1 for _, _, k in self.moves if k == "dup")
+        print(f"pile books to ingest: {len(self.ingest_list)}  "
+              f"files dropped as duplicates: {ndrop}  "
+              f"non-book/broken moved out: {sum(1 for _, _, k in self.moves if k in ('non-book', 'broken-book'))}")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default="/mnt/entropy")
+    ap.add_argument("--meta", required=True)
+    ap.add_argument("--enum", required=True)
+    ap.add_argument("--calibre", required=True)
+    ap.add_argument("--dups", default=None)
+    ap.add_argument("--hold", default="/mnt/entropy/.media-campaign/dup-hold-ebooks-20261104")
+    ap.add_argument("--park", default="/mnt/entropy/NeedsProcessing/_from-ebooks-20261104")
+    ap.add_argument("--plan", required=True)
+    ap.add_argument("--ingest", required=True)
+    ap.add_argument("--exceptions", required=True)
+    args = ap.parse_args()
+    c = Campaign(args)
+    c.ingest_list = []
+    c.load()
+    c.run()
+    c.write()
+
+
+if __name__ == "__main__":
+    main()

scripts/media-music.py

@@ -0,0 +1,445 @@
+#!/usr/bin/env python3
+r"""media-music.py — a plan that re-files the music library into Jellyfin's own shape.
+
+    media-music.py --root DIR --enum TSV --rows TSV [--out TSV] [--exceptions FILE]
+
+Jellyfin's music contract (https://jellyfin.org/docs/general/server/media/music/)
+is not the movie one. Jellyfin reads the metadata out of the file, so a folder is
+an ALBUM and nothing else, every disc of an album belongs in that one folder, the
+tags decide the track order, and a file name only matters when it carries a
+character known to break playback (`< > : " / \ | ? *`) or when a lyric file has
+to match the basename of the track it belongs to. The plan is therefore:
+
+    Music/<album artist>/<Album> (<year>)/NN - <Title>.<ext>
+                       ... the same folder, D.NN - <Title>  when the album has discs
+                       Music/Various Artists/...            when its tracks name
+                                                            more than one artist
+
+The authority is the tag row Jellyfin already read — `scripts/media-census.py
+music` — never the release-pack folder a ripper made. A file with no row is not
+guessed at: it parks under `<root>/_unsorted` with the relative path it has today.
+
+The rules below exist because of what a downloaded discography pack actually
+contains:
+
+ * A cuesheet beside exactly one audio file means the album is one image with a
+   track list inside it. That file keeps its own name: numbering it `01` would
+   claim it is one track, and Jellyfin shows one track either way.
+ * Sidecars — images, cuesheets, EAC logs, checksums, booklets, playlists —
+   follow the album the audio in their own folder belongs to. A folder holding
+   two albums gives a sidecar no single owner, so it parks instead of being
+   guessed into one.
+ * A video container (.mkv, .vob, .ifo, …) inside a music library is not music.
+   Renaming it to .mka/.m4a would tell Jellyfin to index video as audio, so it
+   parks for a human decision.
+ * Playlists move with their album and their CONTENTS are never rewritten: the
+   paths inside an .m3u go stale. Editing a file the operator owns to keep a
+   convenience file tidy is not this tool's job.
+
+Nothing is deleted and nothing is overwritten. The output is a list of rename(2)
+intents for `scripts/media-apply.py`, whose preflight refuses any target that is
+already claimed and whose journal is the rollback.
+"""
+import argparse
+import csv
+import os
+import re
+import sys
+from collections import Counter, defaultdict
+
+# What Jellyfin will index as music. .mp4/.mkv/.webm are deliberately NOT here:
+# the vendor doc says an audio-only stream in those containers is never read as
+# music, and this tool renames for a reason it can prove, not to guess at one.
+AUDIO_EXT = {".flac", ".mp3", ".m4a", ".m4b", ".ogg", ".oga", ".opus", ".wav",
+             ".wma", ".aiff", ".aif", ".ape", ".dsf", ".dff", ".wv", ".mka",
+             ".mpc", ".aac"}
+VIDEO_EXT = {".mkv", ".vob", ".ifo", ".bup", ".mpg", ".mpeg", ".avi", ".mov",
+             ".ts", ".m2ts", ".wmv", ".webm", ".mp4", ".m4v", ".iso", ".img",
+             ".bdmv", ".divx", ".ogm"}
+# A lyric has to carry the basename of its track, so it is named after the audio
+# file and not after itself. .txt counts only when its stem matches an audio stem:
+# a loose .txt in an album folder is a note.
+LYRIC_EXT = {".lrc", ".elrc", ".txt"}
+IMAGE_EXT = {".jpg", ".jpeg", ".png", ".webp", ".gif", ".bmp", ".tiff"}
+# The album-artist strings that mean "compilation", so such an album is not filed
+# under one artist who does not own it.
+VARIOUS = {"various", "various artists", "unknown", "unknown artist", "unknow"}
+# The characters Jellyfin documents as breaking playback. They are dropped, not
+# substituted: a replacement character is a guess at what the title meant.
+BAD_CHARS = set('<>:"/\\|?*')
+TRASH_DIR = re.compile(r"(^|/)(\.AGENTTRASH|\.thumbnails|\.Trash-\d+|@eaDir"
+                       r"|lost\+found)(/|$)")
+
+ARGS = None
+HELD = {}                            # path -> (bytes, ext)
+ROWS = {}                            # path -> the tag row
+PLACED = {}                          # audio path -> the destination the plan gave it
+CLAIMED = set()                      # every destination path the plan has handed out
+MOVES, KEEPS = [], []
+PARKED = defaultdict(list)           # reason -> [path]
+NOTES = defaultdict(list)            # section -> [line]
+CUE_DIRS = set()                     # source dirs holding a .cue
+AUDIO_BY_DIR = defaultdict(set)      # source dir -> audio files a row names
+CHILD_OF = defaultdict(set)          # dir -> the dirs directly below it holding audio
+FOLDERS_SEEN = {}                    # (dir, depth) -> album folders placed below it
+SUBTREE_DEPTH = 2                    # how far below a folder its audio may sit
+ALBUMS = {}                          # (artist, album, year) -> the album group
+
+AMBIGUOUS = object()
+
+
+def dominant(values):
+    """The spelling the tags mostly agree on: the most frequent value, ties broken
+    by sort so the answer never depends on the order rows arrived in."""
+    counts = Counter(values)
+    return min(counts.items(), key=lambda kv: (-kv[1], kv[0]))[0]
+
+
+def rel(path):
+    """The path as seen from the library root."""
+    return os.path.relpath(path, ARGS.root)
+
+
+def safe(name):
+    """A name every client can carry: no reserved character, nothing
+    unprintable, no trailing dot or space, and short enough to rename."""
+    out = "".join(c for c in (name or "")
+                  if c not in BAD_CHARS and (c.isprintable() or c == " "))
+    out = re.sub(r"\s+", " ", out).strip().rstrip(". ")
+    while out and len(out.encode("utf-8", "ignore")) > 200:
+        out = out[:-4].rstrip()
+    return out
+
+
+def assign(src, folder, name, ext, kind, note):
+    """Book one file into `folder` under `name`, renamed if that exact destination
+    is already taken — by another file in the plan, or by a file standing there
+    today. Two files claiming one path is the fault that costs a re-run of a 90k
+    plan, and so is a plan that only works if the renames happen in the right
+    order: the executor refuses any target that is occupied, so an occupied path
+    is never a destination, even when its occupant is scheduled to leave.
+    A file already standing where the contract puts it is a keep, not a move."""
+    stem = safe(name) or "file"
+    dst = os.path.join(folder, stem + ext)
+    if dst == src:
+        CLAIMED.add(dst.lower())          # it is standing on its own target
+        KEEPS.append((src, dst, kind))
+        return dst
+    # A file already in this folder under this name — or under the numbered
+    # variant the collision rule gives it — keeps the name it has. Renumbering it
+    # every run would turn the plan into a shuffle instead of a fixed point.
+    base = os.path.splitext(os.path.basename(src))[0]
+    if os.path.dirname(src) == folder and (base == stem
+                                           or base.startswith(stem + " (")):
+        CLAIMED.add(src.lower())
+        KEEPS.append((src, src, kind))
+        return src
+    n = 1
+    while dst.lower() in CLAIMED:
+        n += 1
+        dst = os.path.join(folder, f"{stem} ({n}){ext}")
+    if n > 1:
+        NOTES[note].append(rel(dst))
+    CLAIMED.add(dst.lower())
+    MOVES.append((src, dst, kind))
+    return dst
+
+
+def park(path, reason):
+    """Hold a file where it is, under the park folder, keeping its relative path."""
+    PARKED[reason].append(path)
+    dst = os.path.join(ARGS.park_dir, rel(path))
+    if dst != path:
+        CLAIMED.add(dst.lower())
+        MOVES.append((path, dst, "park"))
+
+
+def under_park(path):
+    """True for a file already parked: that pile is for a human to review, and
+    re-parking it every run would only push it deeper."""
+    return path.startswith(ARGS.park_dir + os.sep)
+
+
+# ---------------------------------------------------------------------- albums
+def group_albums():
+    """Every tagged track into one group per (album-artist tag, album, year).
+
+    The key is the identity the tags assert, not the folder the file sits in: one
+    album arrives as three discs in three folders, and two copies of an album in
+    one pack are one album to Jellyfin, whose album item is that name, artist and
+    year. The artist in the key is the ALBUM-artist tag and never the performer,
+    because a compilation is one album whose tracks name many performers — keying
+    on the performer would file one album once per artist.
+    """
+    for path, r in sorted(ROWS.items()):
+        if path not in HELD or under_park(path):
+            continue
+        d = os.path.dirname(path)
+        AUDIO_BY_DIR[d].add(path)
+        CHILD_OF[os.path.dirname(d)].add(d)   # so a cover above CD1/CD2 sees both
+        tag = (r["artist"] or "").strip()
+        album = (r["album"] or "").strip()
+        performer = (r["performer"] or "").strip()
+        if not album:
+            park(path, "a track no album names")
+            continue
+        if not tag and not performer:
+            park(path, "a track no artist names")
+            continue
+        key = (tag.casefold(), album.casefold(), (r["year"] or "").strip())
+        g = ALBUMS.setdefault(key, {"tags": [], "albums": [], "year": key[2],
+                                    "members": [], "performers": set(),
+                                    "mb": set()})
+        g["members"].append(path)
+        g["tags"].append(tag)
+        g["albums"].append(album)
+        g["performers"].add(performer or tag)
+        if r.get("mb"):
+            g["mb"].add(r["mb"])
+
+
+def folder_artist(g):
+    """Whose shelf the album sits on. The album-artist tag owns it when it names
+    one; when the tags name only performers, a single performer names the album
+    and several make it a compilation, which belongs to no one of them. The key is
+    case-folded, so the spelling that reaches the folder is the one the tracks
+    mostly carry — see dominant()."""
+    tags = [t for t in g["tags"] if t]
+    if tags:
+        name = dominant(tags)
+        return "Various Artists" if name.casefold() in VARIOUS else name
+    if len({p.casefold() for p in g["performers"]}) == 1:
+        return dominant(g["performers"])
+    return "Various Artists"
+
+
+def album_folder(g):
+    name = safe(dominant(g["albums"])) or "Untitled"
+    return os.path.join(ARGS.root, safe(folder_artist(g)) or "Unknown Artist",
+                        f"{name} ({g['year']})" if g["year"] else name)
+
+
+def is_cue_image(path):
+    """A folder holding this one audio file and a cuesheet is an album image."""
+    d = os.path.dirname(path)
+    return len(AUDIO_BY_DIR.get(d, ())) == 1 and d in CUE_DIRS
+
+
+def number_of(value, width):
+    return f"{int(value):0{width}d}" if str(value or "").isdigit() else ""
+
+
+def place_albums():
+    for key in sorted(ALBUMS):
+        g = ALBUMS[key]
+        folder = album_folder(g)
+        discs = {g0["disc"] for g0 in (ROWS[p] for p in g["members"]) if g0["disc"]}
+        tracks = [int(ROWS[p]["track"]) for p in g["members"]
+                  if str(ROWS[p]["track"] or "").isdigit()]
+        # More than one disc, or a disc numbered past the first: the track number
+        # alone cannot name a file uniquely, so the disc goes in front of it.
+        multi = len(discs) > 1 or any(str(d).isdigit() and int(d) > 1 for d in discs)
+        width = 3 if multi or (max(tracks, default=0) > 99) else 2
+        for path in sorted(g["members"]):
+            r = ROWS[path]
+            ext = HELD[path][1]
+            base = os.path.splitext(os.path.basename(path))[0]
+            if is_cue_image(path):
+                name = safe(base) or "image"
+            else:
+                title = safe(r["title"] or base) or "untitled"
+                num = number_of(r["track"], 2 if multi else width)
+                if multi and num:
+                    num = f"{r['disc'] or '1'}.{num}"
+                name = f"{num} - {title}" if num else title
+            PLACED[path] = assign(path, folder, name, ext, "track",
+                                  "two tracks claimed one name inside one album")
+
+
+# --------------------------------------------------------------------- sidecars
+def folders_below(d, depth):
+    """The album folders the audio in `d`, or within `depth` directories below it,
+    was placed in. Memoised, because every loose file in a big discography folder
+    asks the same question about it."""
+    if (d, depth) not in FOLDERS_SEEN:
+        seen = {os.path.dirname(PLACED[a]) for a in AUDIO_BY_DIR.get(d, ())
+                if a in PLACED}
+        if depth:
+            for child in CHILD_OF.get(d, ()):
+                seen |= folders_below(child, depth - 1)
+        FOLDERS_SEEN[(d, depth)] = seen
+    return FOLDERS_SEEN[(d, depth)]
+
+
+def owner_folder(d):
+    """The album folder that owns this directory — the first place up the tree
+    where audio sits, in the directory itself or within SUBTREE_DEPTH levels below
+    it, which is how a box with its discs in CD1/CD2 owns the cover lying above
+    them. None when no audio is near enough to say, AMBIGUOUS when the audio there
+    belongs to more than one album."""
+    while d.startswith(ARGS.root + os.sep) and d != ARGS.root:
+        folders = folders_below(d, SUBTREE_DEPTH)
+        if folders:
+            return next(iter(folders)) if len(folders) == 1 else AMBIGUOUS
+        d = os.path.dirname(d)
+    return None
+
+
+def place_sidecars():
+    """Everything that is not a tagged track: lyrics, images, cuesheets, logs,
+    booklets, playlists — and the clutter that is none of those."""
+    # The lyric rule needs the audio's stem as it is named TODAY, before the plan
+    # renames it, because that is what the sidecar's own name was copied from.
+    stem = {(d, os.path.splitext(os.path.basename(a))[0]): a
+            for d, audios in AUDIO_BY_DIR.items() for a in audios}
+    for path, (_size, ext) in sorted(HELD.items()):
+        if path in ROWS or under_park(path) or TRASH_DIR.search(path):
+            continue
+        d, base = os.path.dirname(path), os.path.splitext(os.path.basename(path))[0]
+        if ext in VIDEO_EXT:
+            park(path, "a video container inside the music library: Jellyfin will "
+                       "not read it as music, and renaming it would claim it is")
+        elif ext in AUDIO_EXT:
+            park(path, "an audio file no provider row names — its tags are not in "
+                       "the library Jellyfin has matched")
+        elif ext in (LYRIC_EXT | IMAGE_EXT) and (d, base) in stem:
+            # A lyric, or a picture named after its track: it follows that track
+            # and takes the name the track now has, or Jellyfin cannot find it.
+            audio = stem[(d, base)]
+            dst = PLACED.get(audio)
+            if dst:
+                folder, name = os.path.dirname(dst), os.path.splitext(
+                    os.path.basename(dst))[0]
+                assign(path, folder, name, ext, "lyric",
+                       "a sidecar whose name collided with another in its album")
+            else:
+                park(path, "a sidecar of a track the plan could not place")
+        else:
+            owner = owner_folder(d)
+            if owner is AMBIGUOUS:
+                park(path, "a sidecar whose folder holds more than one album, so "
+                           "nothing says which one it belongs to")
+            elif owner:
+                assign(path, owner, base, ext,
+                       "playlist" if ext in (".m3u", ".m3u8") else "sidecar",
+                       "one album folder would have carried two files of the same "
+                       "sidecar name")
+            else:
+                park(path, "not music, or music with nothing beside it to belong to")
+
+
+# ---------------------------------------------------------------------- outputs
+def write_plan():
+    """mkdir rows for every destination folder and its whole chain, then the moves.
+    Parents first: the executor creates directories in the order they are listed."""
+    chain = set()
+    for _s, dst, _k in MOVES:
+        d = os.path.dirname(dst)
+        while d and d != ARGS.root and d != os.path.dirname(d):
+            chain.add(d)
+            d = os.path.dirname(d)
+    dirs = sorted(chain, key=lambda p: (p.count("/"), p))
+    with open(ARGS.out, "w") as fh:
+        fh.write("action\tsrc\tdst\tkind\n")
+        for d in dirs:
+            fh.write(f"mkdir\t\t{d}\tdir\n")
+        for src, dst, kind in sorted(MOVES, key=lambda m: (m[1], m[0])):
+            fh.write(f"move\t{src}\t{dst}\t{kind}\n")
+        for src, dst, kind in sorted(KEEPS, key=lambda m: (m[1], m[0])):
+            fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
+    parked = [m for m in MOVES if m[2] == "park"]
+    sized = sum(HELD[s][0] for s, _d, _k in MOVES if s in HELD)
+    print(f"plan: {len(MOVES) - len(parked)} moves, {len(parked)} parked"
+          f" ({sized / 2**40:.2f} TiB touched), {len(KEEPS)} already placed,"
+          f" {len(dirs)} folders -> {ARGS.out}")
+
+
+def write_exceptions():
+    with open(ARGS.exceptions, "w") as fh:
+        fh.write(f"Music relayout exceptions for {ARGS.root}\n\n")
+        fh.write(f"Everything below is parked under {ARGS.park_dir}, keeping the\n"
+                 "relative path it has today. Nothing is deleted and nothing is\n"
+                 "overwritten: a parked file is one rename from wherever the plan is\n"
+                 "pointed next.\n\n")
+        for reason in sorted(PARKED, key=lambda r: -len(PARKED[r])):
+            rs = PARKED[reason]
+            fh.write(f"{len(rs):>6} files  {reason}\n")
+            for p in sorted(rs, key=rel)[:400]:
+                fh.write(f"          {rel(p)}\n")
+            if len(rs) > 400:
+                fh.write(f"          … and {len(rs) - 400} more\n")
+            fh.write("\n")
+        for section in sorted(NOTES, key=lambda s: -len(NOTES[s])):
+            lines = sorted(set(NOTES[section]))
+            fh.write(f"{len(lines):>6}  {section}\n")
+            for line in lines[:200]:
+                fh.write(f"          {line}\n")
+            if len(lines) > 200:
+                fh.write(f"          … and {len(lines) - 200} more\n")
+            fh.write("\n")
+        no_year = sorted(f"{folder_artist(g)}/{dominant(g['albums'])}"
+                         for g in ALBUMS.values() if not g["year"])
+        if no_year:
+            fh.write(f"{len(no_year):>6} albums carry no year in any tag, so their folders\n"
+                     "       are named without one; Jellyfin still matches them.\n")
+            fh.write("       " + ", ".join(no_year[:300]) + "\n\n")
+        clash = sorted(f"{folder_artist(g)}/{dominant(g['albums'])}"
+                       for g in ALBUMS.values() if len(g["mb"]) > 1)
+        if clash:
+            fh.write(f"{len(clash):>6} albums put one folder name on more than one\n"
+                     "       MusicBrainz release group — different releases, one name.\n"
+                     "       They share the folder Jellyfin's own album item names.\n")
+            fh.write("       " + ", ".join(clash[:200]) + "\n\n")
+        fh.write("       Playlists move with their album and their contents are never\n"
+                 "       rewritten: the paths inside an .m3u go stale.\n")
+    print(f"exceptions -> {ARGS.exceptions}")
+
+
+def main():
+    global ARGS
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default=os.environ.get("MEDIA_MUSIC_ROOT",
+                                                     "/mnt/entropy/Music"))
+    ap.add_argument("--enum", default="/tmp/music-enum.tsv",
+                    help="the enumerate TSV (bytes, ext, path)")
+    ap.add_argument("--rows", default="/tmp/jellyfin-music.tsv",
+                    help="the tag rows scripts/media-census.py music wrote")
+    ap.add_argument("--out", default="/tmp/music-plan.tsv")
+    ap.add_argument("--exceptions", default="/tmp/music-exceptions.txt")
+    ap.add_argument("--park", default="_unsorted",
+                    help="the folder inside the root that holds what cannot be filed")
+    ARGS = ap.parse_args()
+    ARGS.root = os.path.normpath(ARGS.root)
+    ARGS.park_dir = os.path.join(ARGS.root, ARGS.park)
+    read_inputs()
+    # Every path a file occupies today is claimed before a single destination is
+    # handed out: the executor refuses an occupied target, so a plan that aimed at
+    # one would only fail after the renames that were supposed to clear it.
+    CLAIMED.update(p.lower() for p in HELD)
+    for path, (_size, ext) in HELD.items():
+        if ext == ".cue":
+            CUE_DIRS.add(os.path.dirname(path))
+    group_albums()
+    place_albums()
+    place_sidecars()
+    write_plan()
+    write_exceptions()
+
+
+def read_inputs():
+    with open(ARGS.enum) as fh:
+        for line in fh:
+            parts = line.rstrip("\n").split("\t")
+            if len(parts) >= 3:
+                HELD[parts[2]] = (int(parts[0]), parts[1].lower())
+    with open(ARGS.rows, newline="") as fh:
+        for row in csv.DictReader(fh, delimiter="\t"):
+            if row.get("file"):
+                ROWS[row["file"]] = row
+    gone = [p for p in ROWS if p not in HELD]
+    if gone:
+        NOTES["a tag row whose file the census no longer sees"].extend(gone)
+
+
+if __name__ == "__main__":
+    main()

scripts/media-refresh-jellyfin.py

@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+"""Trigger and poll Jellyfin's library scan — the step that makes a media move visible.
+
+    media-refresh-jellyfin.py trigger [--force]
+    media-refresh-jellyfin.py status [--after EPOCH]
+    media-refresh-jellyfin.py wait [--after EPOCH] [--interval S] [--require-quiet N]
+
+`status` answers {"scan": true, "taskState": "Running", "recentMedia": 17}: the
+Scan Media Library row plus the rows the database has taken in recently; with
+`--after` (the trigger's epoch — a file mtime is already passed by an earlier scan) it
+also answers `scannedAfter` and exits non-zero
+while the last finished scan predates it. `wait` calls the library settled only
+on the page's law: a sighting of the task Running — or a run that finished while
+it watched, or a `--after` the task already passes — and THEN `--require-quiet`
+consecutive reads with no scan and no writes. Both the scan and the metadata
+fetch must be quiet, and the Running sighting is what stops a watcher from
+declaring a pre-scan library settled (Dankserver/Jellyfin.md).
+
+Laws this encodes:
+  * Auth is `Authorization: MediaBrowser Token="…"`. A bare X-Emby-Token answers
+    401 (measured on franksdankcloud 12.4.0), which reads exactly like a dead
+    key. The key comes from a mode-600 file and never rides argv or a URL:
+    /proc/<pid>/cmdline is world-readable.
+  * `POST /Library/Refresh` (204) is the ONLY door that queues work on 12.4, and
+    it walks every library. `POST /Items/{id}/Refresh` answers 204 and queues
+    NOTHING (measured twice, 2026-11-02) — so there is no --library here.
+  * A SECOND accepted POST cancels and restarts the scan already running, so
+    progress looks frozen while you hammer it. `trigger` therefore refuses while
+    the Scan Media Library task is not Idle; --force says you want the restart.
+  * 429 means busy. Waiting is the only response that helps, and it is bounded
+    so a wedged server ends the run with a named failure instead of a loop.
+  * `RefreshStatus` on /Library/VirtualFolders stays Idle through a live scan;
+    never poll it. The task table is the only honest read.
+
+Defaults are this fleet's layout: http://127.0.0.1:8096, the key at
+~/.config/secrets/jellyfin, and the database found by trying, in order, $JELLYFIN_DB,
+~/.local/share/jellyfin/jellyfin.db and the quadlet's config bind.
+"""
+
+import argparse
+import json
+import os
+import pathlib
+import sqlite3
+import sys
+import time
+import urllib.error
+import urllib.request
+from datetime import datetime, timezone
+
+SCAN_TASK = "Scan Media Library"
+IDLE = ("Idle",)  # Running, Cancelling and Waiting all mean: not settled
+DB_CANDIDATES = ("~/.local/share/jellyfin/jellyfin.db",
+                 "/home/dank/Containers/jellyfin/config/data/jellyfin.db")
+
+
+def _epoch(iso: str | None) -> float | None:
+    """Jellyfin's `2026-11-04T18:31:33Z` EndTimeUtc as a unix epoch, or None."""
+    if not iso:
+        return None
+    for fmt in ("%Y-%m-%dT%H:%M:%S.%fZ", "%Y-%m-%dT%H:%M:%SZ"):
+        try:
+            return datetime.strptime(iso, fmt).replace(tzinfo=timezone.utc).timestamp()
+        except ValueError:
+            continue
+    sys.exit(f"the server sent an unparsable EndTimeUtc: {iso!r}")
+
+
+def read_key(value: str) -> str:
+    path = pathlib.Path(value).expanduser()
+    try:
+        token = path.read_text().strip()
+    except OSError as exc:
+        sys.exit(f"no Jellyfin API key at {path} ({exc.strerror}); "
+                 "that file is the credential this driver authenticates with")
+    if not token:
+        sys.exit(f"Jellyfin API key file {path} is empty")
+    return token
+
+
+def request(base: str, token: str, method: str, path: str,
+            timeout: float = 30.0) -> tuple[int, bytes]:
+    req = urllib.request.Request(
+        base.rstrip("/") + path, method=method,
+        headers={"Authorization": f'MediaBrowser Token="{token}"'})
+    try:
+        with urllib.request.urlopen(req, timeout=timeout) as resp:
+            return resp.status, resp.read()
+    except urllib.error.HTTPError as exc:
+        return exc.code, exc.read()
+
+
+def scan_task(base: str, token: str) -> dict:
+    """The one task row that means something, and when its last run finished."""
+    code, body = request(base, token, "GET", "/ScheduledTasks")
+    if code != 200:
+        sys.exit(f"GET /ScheduledTasks -> HTTP {code}: {body[:200]!r}")
+    for task in json.loads(body):
+        if task.get("Name") == SCAN_TASK:
+            state = task.get("State")
+            return {"state": state, "running": state not in IDLE,
+                    "lastScan": _epoch(task.get("EndTimeUtc"))}
+    sys.exit(f"the server does not list a {SCAN_TASK!r} task; refusing to guess")
+
+
+def find_db(value: str) -> pathlib.Path:
+    wanted = [value] if value != "-" else []
+    wanted += [p for p in (os.environ.get("JELLYFIN_DB"),) if p]
+    wanted += list(DB_CANDIDATES)
+    for candidate in wanted:
+        path = pathlib.Path(candidate).expanduser()
+        if path.is_file():
+            return path
+    sys.exit("no Jellyfin database found; tried: " + ", ".join(wanted))
+
+
+def recent_media(db: str, window: int) -> int:
+    since = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(time.time() - window * 60))
+    con = sqlite3.connect(f"file:{find_db(db)}?mode=ro", uri=True)
+    try:
+        return con.execute(
+            "SELECT COUNT(*) FROM BaseItems WHERE DateModified >= ?", (since,)
+        ).fetchone()[0]
+    finally:
+        con.close()
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("command", choices=["trigger", "status", "wait"])
+    ap.add_argument("--url", default="http://127.0.0.1:8096")
+    ap.add_argument("--key-file", default="~/.config/secrets/jellyfin")
+    ap.add_argument("--db", default="-", help="jellyfin.db; - searches the known homes")
+    ap.add_argument("--window", type=int, default=5,
+                    help="minutes of DB writes that still count as recent")
+    ap.add_argument("--retries", type=int, default=3, help="extra attempts against a 429")
+    ap.add_argument("--retry-sleep", type=float, default=20.0)
+    ap.add_argument("--force", action="store_true",
+                    help="trigger even though a scan is already running")
+    ap.add_argument("--interval", type=float, default=12.0, help="wait: seconds between reads")
+    ap.add_argument("--require-quiet", type=int, default=3,
+                    help="wait: consecutive quiet reads before the library counts as settled")
+    ap.add_argument("--after", type=float, default=None,
+                    help="status/wait: an epoch the scan must have finished after — pass "
+                         "the trigger's, not a file mtime an earlier scan already passes")
+    ap.add_argument("--start-window", type=float, default=120.0,
+                    help="wait: seconds to allow the scan to start before reporting "
+                         "that the 204 queued nothing")
+    ap.add_argument("--timeout", type=float, default=1800.0, help="wait: seconds before giving up")
+    args = ap.parse_args()
+    token = read_key(args.key_file)
+
+    def state() -> dict:
+        task = scan_task(args.url, token)
+        out = {"scan": task["running"], "taskState": task["state"],
+               "lastScan": task["lastScan"],
+               "recentMedia": recent_media(args.db, args.window)}
+        if args.after is not None:
+            out["scannedAfter"] = bool(task["lastScan"] and task["lastScan"] >= args.after)
+        return out
+
+    if args.command == "status":
+        current = state()
+        print(json.dumps(current))
+        if args.after is not None and not current["scannedAfter"]:
+            sys.exit(f"the last scan finished before {args.after:.0f}; "
+                     "a refresh is still owed")
+        return
+
+    if args.command == "wait":
+        start = time.monotonic()
+        ended_at_entry, quiet, started, first = None, 0, False, True
+        while True:
+            current = state()
+            if first:
+                first, ended_at_entry = False, current["lastScan"]
+            if (current["scan"]
+                    or (current["lastScan"] and current["lastScan"] != ended_at_entry)
+                    or current.get("scannedAfter")):
+                # The settle law: a Running sighting (or a run that finished while
+                # watching, or a --after the task already passes) must come BEFORE
+                # any claim that the library is quiet.
+                started = True
+            busy = current["scan"] or current["recentMedia"]
+            quiet = quiet + 1 if started and not busy else 0
+            print(json.dumps(current | {"quiet": quiet}), flush=True)
+            if quiet >= args.require_quiet:
+                return
+            elapsed = time.monotonic() - start
+            if not started and elapsed >= args.start_window:
+                sys.exit(f"{SCAN_TASK} never started: it read {current['taskState']!r} "
+                         f"for {elapsed:.0f}s and the database took no writes. Either "
+                         "the POST queued nothing — a POST that arrives while a scan "
+                         "is already running cancels it, and every later 204 is "
+                         "accepted and discarded, so restart the service and trigger "
+                         "once — or the scan finished before this watch began, in "
+                         "which case ask with --after <journal mtime> instead.")
+            if elapsed >= args.timeout:
+                sys.exit(f"still busy after {args.timeout:.0f}s: {current}")
+            time.sleep(args.interval)
+
+    if scan_task(args.url, token)["running"] and not args.force:
+        sys.exit(f"{SCAN_TASK} is already running on {args.url}: a second "
+                 "POST /Library/Refresh cancels and restarts it, so the index "
+                 "looks frozen while you wait. Poll `wait` instead of triggering "
+                 "again (--force overrides).")
+
+    for attempt in range(args.retries + 1):
+        code, body = request(args.url, token, "POST", "/Library/Refresh")
+        if code == 204:
+            print("refresh accepted (204); the scan walks every library")
+            return
+        if code == 429:
+            if attempt < args.retries:
+                time.sleep(args.retry_sleep)
+                continue
+            sys.exit(f"Jellyfin is busy: {args.retries + 1} attempts all answered "
+                     "429. Nothing was queued; run `wait`, then trigger again.")
+        sys.exit(f"POST /Library/Refresh -> HTTP {code}: {body[:200]!r}")
+
+
+if __name__ == "__main__":
+    main()

scripts/media-refresh-plex.sh

@@ -0,0 +1,130 @@
+#!/usr/bin/env bash
+# Trigger and poll ONE Plex library section — the step that makes a media move visible.
+#
+#   media-refresh-plex.sh trigger --library KEY
+#   media-refresh-plex.sh status  --library KEY [--after EPOCH]
+#
+# One section only: refreshing all five costs a full walk of each and buys
+# nothing for a rename that touched one tree. And a rename on the array is
+# invisible until the section is rescanned — the server is bound at the SSD
+# metadata view, so an array-side move is not an event its watcher ever sees.
+#
+# Two host realities shape this script:
+#   * PMS's HTTPD closes without a single byte on a request that arrives from
+#     another netns (the host and the 10.89.x.x container address both read as
+#     "empty reply"), and it serves TLS to remote clients. So the request rides
+#     plain http://127.0.0.1:32400 INSIDE the container, shipped in as a file —
+#     a three-layer ssh/podman/sh quoting chain only produces parse errors.
+#   * The token is `PlexOnlineToken` in the container's Preferences.xml (not
+#     PlexHTTPToken, which a claimed server may not carry). It is read inside
+#     the container and used from a curl config there: never argv (any process
+#     on the host reads /proc/<pid>/cmdline), never a URL (proxies log URLs).
+#
+# Progress needs no token and no container: `section_locations.scanned_at` in
+# the library database is the same epoch the API prints as `scannedAt`
+# (verified 2026-11-04), so `status` reads the database and compares it against
+# --after — which makes it a census of the last RECORDED scan, never a waiter: PMS
+# stamps that epoch when it accepts the request, so the door answers `settled` in the
+# same second its own `trigger` ran (measured 2026-11-04; no epoch survives it).
+# `trigger` answering 000 means PMS is still warming — retry, never
+# restart the service over it. Proof that a rename actually landed is
+# scripts/media-verify.py replaying the journal; this script only moves the
+# scanner along.
+set -euo pipefail
+
+CONTAINER=plex
+URL="http://127.0.0.1:32400"
+PREFS='/config/Library/Application Support/Plex Media Server/Preferences.xml'
+DB="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db"
+SECTION=""
+AFTER=""
+TRIES=6
+WARM_SLEEP=10
+
+die() { echo "$*" >&2; exit 1; }
+
+CMD="${1:-}"; shift || true
+case "$CMD" in
+    trigger|status) ;;
+    *) die "usage: $(basename "$0") trigger|status --library KEY [--after EPOCH] [--container NAME] [--db PATH]" ;;
+esac
+
+while [ $# -gt 0 ]; do
+    case "$1" in
+        --library)  SECTION="${2:-}"; shift 2 ;;
+        --after)    AFTER="${2:-}"; shift 2 ;;
+        --tries)    TRIES="${2:-}"; shift 2 ;;
+        --warm-sleep) WARM_SLEEP="${2:-}"; shift 2 ;;
+        --container) CONTAINER="${2:-}"; shift 2 ;;
+        --url)      URL="${2:-}"; shift 2 ;;
+        --prefs)    PREFS="${2:-}"; shift 2 ;;
+        --db)       DB="${2:-}"; shift 2 ;;
+        *) die "unknown argument: $1" ;;
+    esac
+done
+
+[ -n "$SECTION" ] || die "--library is required (the section id from GET /library/sections; Movies is 1)"
+command -v python3 >/dev/null || die "python3 is required to read the library database"
+
+if [ "$CMD" = "status" ]; then
+    [ -r "$DB" ] || die "no readable Plex library database at $DB"
+    python3 - "$DB" "$SECTION" "$AFTER" <<'PY'
+import datetime, sqlite3, sys
+
+db, section, after = sys.argv[1], int(sys.argv[2]), sys.argv[3]
+con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+rows = con.execute(
+    "SELECT root_path, available, scanned_at FROM section_locations "
+    "WHERE library_section_id = ? ORDER BY id", (section,)).fetchall()
+if not rows:
+    sys.exit(f"section {section} has no location in {db}; is the id right?")
+worst = min(r[2] or 0 for r in rows)
+for root, available, at in rows:
+    iso = (datetime.datetime.fromtimestamp(at, datetime.UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
+           if at else "never")
+    print(f"  available={available} scanned_at={iso} {root}")
+print(f"section={section} scanned_at={worst}")
+if after:
+    if worst < int(after):
+        sys.exit(f"section {section} was last scanned before {after}; a refresh is still owed")
+PY
+    exit 0
+fi
+
+command -v podman >/dev/null || die "podman is required to reach PMS inside its container"
+
+INNER="$(mktemp -t pxscan.XXXXXX)"
+trap 'rm -f "$INNER"' EXIT
+cat > "$INNER" <<'EOS'
+# Runs INSIDE the plex container: $1 section key, $2 base URL, $3 Preferences.xml.
+key="$1"; url="$2"; prefs="$3"
+[ -r "$prefs" ] || { echo "no readable $prefs inside the container" >&2; exit 3; }
+T=$(sed -n 's/.*PlexOnlineToken="\([^"]*\)".*/\1/p' "$prefs" | head -1)
+[ -n "$T" ] || { echo "no PlexOnlineToken in $prefs" >&2; exit 3; }
+cfg=$(mktemp)
+( umask 077
+  { printf 'header = "X-Plex-Token: %s"\n' "$T"
+    printf 'silent\nshow-error\nrequest = "POST"\n'
+    printf 'url = "%s/library/sections/%s/refresh"\n' "$url" "$key"; } > "$cfg" )
+code=$(curl -K "$cfg" -o /dev/null -w '%{http_code}' || true)
+shred -u "$cfg" 2>/dev/null || rm -f "$cfg"
+[ "$code" = "401" ] && { echo "section $key: HTTP 401 — the token in $prefs is not accepted" >&2; exit 4; }
+echo "$code"
+EOS
+
+attempt=1
+while :; do
+    podman cp "$INNER" "$CONTAINER:/tmp/media-refresh-plex.sh" \
+        || die "could not copy the runner into container $CONTAINER"
+    code="$(podman exec "$CONTAINER" sh /tmp/media-refresh-plex.sh "$SECTION" "$URL" "$PREFS" || true)"
+    podman exec "$CONTAINER" rm -f /tmp/media-refresh-plex.sh >/dev/null 2>&1 || true
+    case "$code" in
+        204|200) echo "refresh accepted ($code) for section $SECTION"; exit 0 ;;
+        000)
+            [ "$attempt" -ge "$TRIES" ] && die "PMS never answered after $TRIES attempts (000); it may still be warming — do not restart the service over it"
+            echo "section $SECTION: no answer yet ($attempt/$TRIES), PMS is warming" >&2 ;;
+        *) die "refresh of section $SECTION failed: HTTP ${code:-curl-error}" ;;
+    esac
+    attempt=$((attempt + 1))
+    sleep "$WARM_SLEEP"
+done

scripts/media-shows-tvdb-ids.tsv

@@ -0,0 +1,35 @@
+# The operator's tvdb id map: `folder <TAB> tvdb id <TAB> note`.
+#
+# These are the pack folders whose show tvdb's NAME SEARCH cannot reach: the
+# record exists, but tvdb titles it in the show's own language and the English
+# name rides in aliases the search never reads. Without a row here the folder
+# keeps the release folder's words and no episode title; with one, both matchers
+# read `[tvdbid-NNNN]` and the files rename to tvdb's own name and titles.
+#
+# Every id was read from tvdb's own record (`/series/<id>/extended`) on
+# 2026-10-31, and the note states what that record says plus the neighbouring
+# record a name search would have picked instead — the row is checkable without
+# re-searching, because one wrong id renames a whole pack to the wrong show. The
+# plan refuses a line whose id is not a number and lists every folder the map
+# touched in the exceptions file.
+#
+# Folders the name search DOES reach (the EXACT and PUNCT classes of the proposal
+# table) stay off this file on purpose: a map row would hide a live lookup behind
+# a number nobody re-checks.
+[Anime Time] Pokemon Complete Series (Season 01-25+Movies+Specials+Shorts+OST) [DVD+BD][1080p][HEVC 10bit x265][AAC]	76703	tvdb: ポケットモンスター (1997), the umbrella record for every era this pack spans; 1268 files
+[Anime Time] One Piece (0001-1071+Movies+Specials) [BD+CR] [Dual Audio] [1080p][HEVC 10bit x265][AAC][Multi Sub]	81797	tvdb: ワンピース (1999), alias ONE PIECE; the bare alias "One Piece" belongs to 392276, the 2023 live action; 1072 files
+[Judas] Naruto (Complete Series) [BD 1080p][HEVC x265 10bit][Multi-Audio][Multi-Subs]	78857	tvdb: NARUTO－ナルト－ (2002), alias "Naruto" carries the full-width dashes; 220 files
+Star Trek Deep Space 9 Deluxe DVD Boxset + Extras DVDRip TSV	72073	tvdb: Star Trek: Deep Space Nine (1993), alias "Star Trek Deep Space 9"; the box set's own words match nothing; 279 files
+Top Gear UK 1-17	74608	tvdb: Top Gear (2002), no "UK" alias — the US version is 199051; 131 files
+Black Clover 001	331753	tvdb: ブラッククローバー (2017), which holds NO alias at all; 378762 is Mugyutto! Black Clover (2019), a different show; 108 files
+Shin Chan - Complete Eng Dub Series 2160p Ai Upscaled -Mesc	437356	tvdb: Shin Chan (2006) — the English dub record this pack is; 79654 is the 1992 Japanese original; 78 files
+Fairy Tail (2009-2014) (Seasons 1-8 + OVAs) (BD 1080p)(HEVC x265 10bit)(Dual-Audio)(Eng-Subs)-Judas[TGx]	114801	tvdb: フェアリーテイル (2009), alias FAIRY TAIL; 410031 is FAIRY TAIL 100 NEN QUEST (2024), the sequel; 59 files
+(CBB) Tokyo Ghoul - The Complete Series (BD 1080p)(Dual-Audio)(HEVC-10bit-x265)	281249	tvdb: 東京喰種トーキョーグール (2014) — :re and √A are seasons of THIS record, confirmed by search; 58 files
+[Anime Time] Demon Slayer - Kimetsu no Yaiba (S01+S02+S03+S04+Mugen Train Arc+Movie) [BD] [Dual Audio][1080p][HEVC 10bit x265][AAC][Eng Sub]	348545	tvdb: 鬼滅の刃 (2019), alias "Demon Slayer: Kimetsu no Yaiba" — the folder's hyphen splits it differently; 56 files
+(CBB) Tengen Toppa Gurren Lagann - The Complete Series (BD 1080p)(Dual-Audio)(HEVC-x265-10bit)	80096	tvdb: 天元突破グレンラガン (2007), alias Tengen Toppa Gurren Lagann; 51 files
+Fullmetal Alchemist - Brotherhood	85249	tvdb: 鋼の錬金術師 FULLMETAL ALCHEMIST (2009), alias "Full Metal Alchemist Brotherhood" — spaced, unlike this folder; 64 files
+[NewbSubs] Attack on Titan~Shingeki no Kyojin Series (1080p Blu-ray x265 Dual Audio)	267440	tvdb: 進撃の巨人 (2013), aliases carry "Attack on Titan" and "Shingeki no Kyojin", joined here by a tilde; 27 files
+Kimetsu no Yaiba	348545	the same record as the Anime Time pack above: two release folders, ONE show; 12 files
+Rick and Morty The Anime	420279	tvdb: リック・アンド・モーティ：ジ・アニメ (2024), no English alias on the record; 10 files
+[Anime Time] Dorohedoro (Season 1+OVA+Extra) [Dual Audio][BD][1080p][HEVC 10bit x265][AAC][Eng Sub]	370761	tvdb: ドロヘドロ (2020), alias Dorohedoro; 9 files
+Ken Il Guerriero HOKUTO NO KEN	79156	tvdb: 北斗の拳 (1984), alias "Hokuto no Ken"; the Italian title matches nothing and 465216 is the 2026 remake; 5 files

scripts/media-shows-tvdb.py

@@ -0,0 +1,306 @@
+#!/usr/bin/env python3
+r"""Ask tvdb what it calls the shows a plan renamed WITHOUT a tvdb id.
+
+ media-shows-tvdb.py [--plan TSV] [--root DIR] [--park DIR] [--map TSV]
+                     [--out TSV] [--credentials FILE]
+
+`media-shows.py` renames a series to tvdb's own words only when an id backs the
+name; everything else keeps the metadata provider's words and no episode title,
+and the plan's mkdir rows are where those folders show up. This script turns that
+list into a decision sheet: one row per destination series folder that carries no
+`[tvdbid-NNNN]`, with what tvdb's search answers for it and a verdict that says
+HOW MUCH was guessed to get there.
+
+It changes nothing. The sheet is for the operator; the file that changes
+behaviour is `scripts/media-shows-tvdb-ids.tsv`, whose rows he approves. That is
+the loop: plan -> this sheet -> approved map rows -> re-plan.
+
+Verdicts, weakest last:
+
+  EXACT   tvdb's own name equals the folder's words as they stand.
+  PUNCT   it equals them only after release junk is stripped (pack numbers,
+          codecs, group names, `Complete Series`); the difference is spelling,
+          not identity.
+  NEAR    tvdb answered something whose name is NOT the query. The best hit is
+          printed so a human can judge it, and it is never applied by anything.
+  NONE    nothing left to ask with, or no answer at all.
+
+Acceptance is deliberately unforgiving — a hit counts only when tvdb's name
+equals the query with every non-alphanumeric dropped from both sides, and the
+series' year still has to fall inside any year span the folder names. The
+cleaner is the only thing that got smarter over the passes this tool replaces;
+the acceptance test never moved, because a wrong id renames a whole pack to the
+wrong show. A query that needs the cleaner is reported as PUNCT for exactly that
+reason: the machine stripped words, so a human reads the row.
+
+Read-only against tvdb. The credential file is the planner's own
+(`api_key=`, optional `pin=`), read here and never printed.
+"""
+import argparse
+import collections
+import importlib.util
+import json
+import os
+import re
+import sys
+import time
+import urllib.error
+import urllib.parse
+import urllib.request
+
+ARGS = argparse.ArgumentParser(description=__doc__.splitlines()[0],
+                               formatter_class=argparse.RawDescriptionHelpFormatter)
+ARGS.add_argument("--plan", default="/tmp/tv-plan.tsv",
+                  help="the plan media-shows.py just wrote (its mkdir rows are "
+                       "the folders this sheet asks about)")
+ARGS.add_argument("--root", default=os.environ.get("TV_ROOT", "/mnt/entropy/TVShows"),
+                  help="the TV library the plan renames")
+ARGS.add_argument("--park", default="",
+                  help="the review pile inside it (its folders are not shows)")
+ARGS.add_argument("--map",
+                  default=os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                                       "media-shows-tvdb-ids.tsv"),
+                  help="the operator's approved map; its folders are reported as "
+                       "MAPPED and never asked again")
+ARGS.add_argument("--credentials",
+                  default=os.path.expanduser("~/.config/secrets/tvdb"),
+                  help="file with an api_key= line (and a pin= for a user-issued "
+                       "key); mode 600, and never printed")
+ARGS.add_argument("--out", default="/tmp/tvdb-proposal.tsv",
+                  help="the decision sheet")
+ARGS = ARGS.parse_args()
+
+ROOT = os.path.abspath(ARGS.root)
+PARK = os.path.abspath(ARGS.park or os.path.join(ROOT, "_unsorted"))
+
+_spec = importlib.util.spec_from_file_location(
+    "eng", os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                        "media-titles.py"))
+eng = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(eng)
+
+# Release metadata, not the show: everything a scene folder name says about the
+# ENCODING, the packaging or the languages, which tvdb has never heard of.
+JUNK = re.compile(
+    r"complete|series|dual|multi|audio|sub|subs|multisub|dualaudio|"
+    r"\d{3,4}p|1080p|720p|2160p|480p|4k|hd|hdtv|cam|hdcam|telesync|x264|x265|hevc|avc|"
+    r"\d+bit|bit|bluray|blu-?ray|bdrip|web-?rip|web-?dl|webrip|hdtvrip|"
+    r"dvdrip|dvdr|dvdscr|tsr|tsv|mkv|avi|mp4|aac|ac3|dts|eac3|flac|ddp?5[.1]1|"
+    r"bd|dvd|rip|boxset|box|season|seasons|episodes?|specials?|shorts|ost|"
+    r"ova|ovas|oad|repack|proper|internal|mi?xed|budgetbits|deluxe|remux|"
+    r"remastered|uncut|collector|edition|extras?|volume|volumes|part|"
+    r"disc|disk|cd|ntsc|pal|region|fixed|patched|proof|english|esp|"
+    r"espagnol|french|vostfr|subtitul\w*|spanish|german|italian|ita|eng|fre|spa|lat|"
+    r"upscaled?|ai|dd|movie|movies",
+    re.I)
+GROUP = re.compile(r"KONTRAST|TGx|Judas|Silence|FreetheFilms|NTb|DEADPOOL|Xploit|"
+                   r"SM737|TSA|3OISL3R|PLAYWEB|EPiC|FLUX|HONEY|Zaxton|Cca3rW|"
+                   r"monikeres|frank|hdh|psi|rtfpemaker|cbm|web|asser|astro|"
+                   r"yfn|uindex|www|org|com|net|mesc|ataraxiaprime|seph\d*|"
+                   r"[a-z]+subs?|edge", re.I)
+BRACKET = re.compile(r"[\[({]([^\])}]*)[\])}]")
+NUM_RANGE = re.compile(r"(?<!\w)\d{1,4}\s*(?:-|–|to|through)\s*\d{1,4}(?!\w)", re.I)
+INSIDE = re.compile(r"all|complete|full|dual|multi|audio|sub|subs|ova|ovas|"
+                    r"movies?|episodes?|extras?|bonus|pack|remux|region\s*\d*|"
+                    r"bd|dvd|cd|ntsc|pal|hr-?dvd", re.I)
+
+
+def drop_bracket(m):
+    """A bracket says nothing when its words are junk; keep it when they aren't."""
+    toks = [t for t in re.split(r"[\s_.,+~&]+", m.group(1)) if t]
+    if not toks or all(INSIDE.fullmatch(t) or JUNK.fullmatch(t)
+                       or GROUP.fullmatch(t)
+                       or re.fullmatch(r"\d+[a-z]?", t, re.I)
+                       or re.fullmatch(r"(19|20)\d\d", t) for t in toks):
+        return " "
+    return " " + m.group(1) + " "
+
+
+def clean(name):
+    """The folder's words with everything about the RELEASE taken out, so what is
+    left is what a person would type into tvdb. `Star Trek Deep Space 9 Deluxe
+    DVD Boxset + Extras DVDRip TSV` becomes `Star Trek Deep Space 9`; `SNL 16-20`
+    becomes `SNL`."""
+    name = re.sub(r"^\s*www\b.*?\b(org|com|net)\b\s*[-–]\s*", " ", name, flags=re.I)
+    name = re.sub(r"\bS\d{1,2}([ .Ee]\d+)?[-–Ee\d.]*", " ", name)
+    name = re.sub(r"\b\d{1,2}[xX]\d{1,3}\b", " ", name)
+    name = NUM_RANGE.sub(" ", name)
+    name = re.sub(r"(19|20)\d\d(?:\s*[-/–]\s*(?:19|20)\d\d)?", " ", name)
+    name = re.sub(r"\bby\s+[A-Za-z0-9_.-]+$", " ", name)
+    name = BRACKET.sub(drop_bracket, name)
+    name = re.sub(r"[\[()\]:,;!?*]", " ", name)
+    toks = re.split(r"[\s_.\-+~&]+", name)
+    out = []
+    for i, t in enumerate(toks):
+        if not t or JUNK.fullmatch(t) or GROUP.fullmatch(t):
+            continue
+        if re.fullmatch(r"\d+", t):
+            after = sum(1 for x in toks[i + 1:] if x
+                        and not JUNK.fullmatch(x) and not GROUP.fullmatch(x)
+                        and not re.fullmatch(r"\d+", x))
+            # A pack index (`001`) and a lead-in number over a real title
+            # (`01 Charles Barkley`) are noise; `30 Rock`, `24`, `Deep Space 9`
+            # are the title itself.
+            if len(t) >= 3 or (not out and after >= 2):
+                continue
+        out.append(t)
+    while out and out[-1].lower() in ("and", "with", "the", "a", "an", "to", "of"):
+        out.pop()
+    return " ".join(out).strip()
+
+
+def years(name):
+    return [int(y) for y in re.findall(r"\b(19\d\d|20\d\d)\b", name)]
+
+
+def bare(s):
+    return re.sub(r"[^a-z0-9]", "", s.lower())
+
+
+CRED = {}
+try:
+    for _line in open(ARGS.credentials):
+        if "=" in _line:
+            k, v = _line.strip().split("=", 1)
+            CRED[k.strip()] = v.strip()
+except OSError:
+    print(f"tvdb: no credentials at {ARGS.credentials} — the sheet would be all "
+          "NEAR and NONE, which is not a decision sheet", file=sys.stderr)
+    sys.exit(2)
+if not CRED.get("api_key"):
+    print(f"tvdb: no api_key in {ARGS.credentials}", file=sys.stderr)
+    sys.exit(2)
+
+TOKEN = {"jwt": ""}
+
+
+def call(path):
+    """One tvdb v4 GET, logging in again once when the JWT expires mid-run."""
+    for attempt in (1, 2):
+        if not TOKEN["jwt"]:
+            body = {"apiKey": CRED["api_key"]}
+            if CRED.get("pin"):
+                body["token"] = CRED["pin"]
+            req = urllib.request.Request("https://api4.thetvdb.com/v4/login",
+                                         data=json.dumps(body).encode(),
+                                         headers={"Content-Type":
+                                                  "application/json"})
+            TOKEN["jwt"] = json.load(
+                urllib.request.urlopen(req, timeout=30))["data"]["token"]
+        try:
+            req = urllib.request.Request("https://api4.thetvdb.com/v4" + path,
+                                         headers={"Authorization":
+                                                  "Bearer " + TOKEN["jwt"]})
+            return json.load(urllib.request.urlopen(req, timeout=30))["data"]
+        except urllib.error.HTTPError as e:
+            if e.code == 401 and attempt == 1:
+                TOKEN["jwt"] = ""
+                continue
+            raise
+
+
+MAPPED = {}
+if ARGS.map and os.path.exists(ARGS.map):
+    for _line in open(ARGS.map):
+        _line = _line.rstrip("\n")
+        if not _line.strip() or _line.startswith("#"):
+            continue
+        cols = _line.split("\t")
+        if len(cols) > 1 and cols[1].strip().isdigit():
+            MAPPED[cols[0].strip()] = cols[1].strip()
+
+# The folders the plan created inside the library, with the file count each one
+# will hold. The review pile is not a show: its files belong to no identity, and
+# naming the pile would put one junk series on the sheet for all of them.
+shows = collections.Counter()
+for line in open(ARGS.plan).read().splitlines()[1:]:
+    parts = line.split("\t")
+    if len(parts) < 4 or parts[0] != "mkdir":
+        continue
+    dst = os.path.abspath(parts[2])
+    if not dst.startswith(ROOT + os.sep) or dst.startswith(PARK + os.sep):
+        continue
+    top = os.path.relpath(dst, ROOT).split(os.sep)[0]
+    if top.startswith((".", "_")):
+        continue
+    if "[tvdbid-" in top:
+        continue                                # an id already backs it
+    shows.setdefault(top, 0)
+for line in open(ARGS.plan).read().splitlines()[1:]:
+    parts = line.split("\t")
+    if len(parts) < 4 or parts[0] != "move":
+        continue
+    dst = os.path.abspath(parts[2])
+    if not dst.startswith(ROOT + os.sep) or dst.startswith(PARK + os.sep):
+        continue
+    top = os.path.relpath(dst, ROOT).split(os.sep)[0]
+    if "[tvdbid-" in top:
+        continue
+    shows[top] += 1
+
+tally = collections.Counter()
+rows = []
+for folder in sorted(shows, key=lambda f: (-shows[f], f)):
+    files, tag, note = shows[folder], "NONE", ""
+    if folder in MAPPED:
+        rows.append([files, "MAPPED", folder, "", "", "", MAPPED[folder],
+                     "the operator's map already states this id"])
+        tally["MAPPED"] += 1
+        continue
+    query, untouched = clean(folder), clean(folder) == folder
+    ys = years(folder)
+    if not query:
+        rows.append([files, "NONE", folder, "", "", "", "",
+                     "nothing is left of the folder's name once release junk"
+                     " is gone"])
+    else:
+        try:
+            hits = call("/search?query=" + urllib.parse.quote(query)
+                        + "&type=series") or []
+        except Exception as e:                                  # noqa: BLE001
+            hits = []
+            note = f"tvdb did not answer: {e}"
+        want = bare(query)
+        pick = None
+        for h in hits:
+            if bare(h.get("name") or "") != want:
+                continue
+            hy = str(h.get("year") or "")
+            if ys and hy.isdigit() and not (min(ys) <= int(hy) <= max(ys)):
+                continue
+            pick = h
+            break
+        if pick:
+            tag = "EXACT" if untouched else "PUNCT"
+            rows.append([files, tag, folder, query, pick.get("name", ""),
+                         str(pick.get("year", "") or ""),
+                         str(pick.get("tvdb_id", "") or ""), ""])
+        elif hits:
+            best = hits[0]
+            rows.append([files, "NEAR", folder, query, best.get("name", ""),
+                         str(best.get("year", "") or ""),
+                         str(best.get("tvdb_id", "") or ""),
+                         f"{len(hits)} answer(s); tvdb's own name is not the "
+                         "query, so nothing would take it"])
+        else:
+            rows.append([files, "NONE", folder, query, "", "", "",
+                         note or "tvdb has no series by that name"])
+    tally[rows[-1][1]] += 1
+    time.sleep(0.12)
+
+# Two folders mapping to one record is not two shows; it is one show the release
+# folders split. Say it on every row involved, or the operator approves the same
+# id twice and reads the second as a mistake.
+by_id = collections.Counter(r[6] for r in rows if r[6] and r[1] != "MAPPED")
+for r in rows:
+    if r[6] and r[1] != "MAPPED" and by_id[r[6]] > 1:
+        r[7] = (f"MERGE: {by_id[r[6]]} folders map to tvdb {r[6]} — one show, "
+                "the releases split it; " + r[7]).strip("; ")
+
+with open(ARGS.out, "w") as fh:
+    fh.write("files\tverdict\tfolder\tquery\ttvdb_name\ttvdb_year\ttvdb_id\tnote\n")
+    for r in rows:
+        fh.write("\t".join(str(c) for c in r) + "\n")
+print(f"sheet: {len(rows)} folders"
+      + "".join(f" {t}={n}" for t, n in sorted(tally.items()))
+      + f" -> {ARGS.out}")

scripts/media-titles.py

@@ -275,7 +275,14 @@ def is_extra(path, title="", dur=0.0, root=ROOT, matched=False):
     bonus_folder = False
     for folder in parts[head:-1]:
         if folder_is_bonus(folder):
-            bonus_folder = not mentions(name_words(name), name_words(folder))
+            # The folder's words are often the film's own name ("Angel's Egg
+            # Extras/"), and then they are not bonus evidence. Only WORDS can name
+            # a film: "extras" is release junk that name_words drops, so a folder
+            # like "01. Extras" reduces to its index — and an index matching an
+            # index ("NCOP 01") is a coincidence, not a title.
+            nw = [w for w in name_words(name) if not w.isdigit()]
+            fw = [w for w in name_words(folder) if not w.isdigit()]
+            bonus_folder = not fw or not mentions(nw, fw)
             break
     if dur and dur > SHORT_MAX:
         # Runtime vetoes vocabulary, with ONE exception: an explicit bonus folder

scripts/media-tv.py

@@ -17,13 +17,20 @@ the executor's own path guard still has teeth, and give it its own journal —
 one journal is one undo, and a TV move must not be welded to a movie relayout:
 
     python3 scripts/media-tv.py --root /mnt/entropy/Movies \
-        --library /mnt/entropy/TVShows --out /mnt/entropy/media-tv-plan.tsv
-    python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
-        --root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
-    python3 scripts/media-apply.py verify --plan /mnt/entropy/media-tv-plan.tsv \
-        --root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
+        --library /mnt/entropy/TVShows \
+        --out /mnt/entropy/.media-campaign/media-tv-plan.tsv
+    python3 scripts/media-apply.py apply \
+        --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
+    python3 scripts/media-apply.py verify \
+        --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
     python3 scripts/media-apply.py sweep --root /mnt/entropy \
-        --journal /mnt/entropy/media-relayout-tv.jsonl --apply
+        --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
+
+Plans and journals live in `/mnt/entropy/.media-campaign/`, never loose on the
+array root: `media-apply.py` defaults its journal to `<root>/media-relayout.jsonl`,
+so `--journal` is always explicit here.
 
 Guards, and why each one is here:
 

scripts/media-tvdupes.py

@@ -0,0 +1,475 @@
+#!/usr/bin/env python3
+"""media-tvdupes.py — find duplicate TV episodes and plan their staging, one
+EPISODE at a time.
+
+A TV library grows the same show into several folders: a tagged season box, an
+untagged packager drop, a `Complete` folder that is not complete. The folders
+look like copies of each other and are not, so the unit of comparison is the
+episode and never the folder. A file is droppable only when EVERY episode it
+carries is also held, somewhere else in the same show, by a file at least as
+large — Frank's rule: the same episode in two places keeps the larger one. A
+file that holds one episode nothing else has stays, whatever else it carries.
+
+It writes a plan and moves nothing; `scripts/media-apply.py` is the only thing
+that renames. The plan stages into a dated hold, so the pass is undoable from
+the journal and frees no space until the hold is purged.
+
+    python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
+        --hold /mnt/entropy/.AGENTTRASH/20261102-tv-duplicates \
+        --out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv
+    python3 scripts/media-apply.py apply \
+        --plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl \
+        --metadata-view /var/lib/media-meta/view --apply
+    python3 scripts/media-apply.py verify \
+        --plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl
+    python3 scripts/media-apply.py sweep --root /mnt/entropy \
+        --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl --apply  # empty folders only
+    scripts/media-view.sh reap                           # ghost sidecars above
+
+Then both servers have to be told: the plan moves files on the raw array, and no
+watcher crosses the metadata view — [[Media Relayout]] owns the refresh order.
+
+Guards, and why each one is here:
+
+* **Identity before geometry.** Two folders holding the same words are not the
+  same show: the 2026-11-02 census found `Avatar: The Last Airbender` (1999,
+  tvdb 74852) beside the 2024 remake (385925), `Star Trek: Voyager` beside
+  `Star Trek: The Animated Series`, and a German thriller Jellyfin had titled
+  `Millennium (2022) [tvdbid-472933]` beside `Millennium (1996)`. A folder's
+  identity is read from three places, in this order: its `[tvdbid-N]` tag, the
+  `tvshow.nfo` Jellyfin wrote — which lives in the metadata view, not on the
+  array, and is what `--meta` reads — and a `folder<TAB>id` map passed with
+  `--ids`. The tag outranks the derived layers: an id that contradicts a folder's
+  own tag is reported and ignored, never unioned — measured 2026-11-02, Jellyfin's
+  provider table carried a 2026 special's id on a folder tagged `[tvdbid-73838]`,
+  and unioning them makes one show of the 2000 series and the special, whose bigger
+  files then dominate the original's pilot. Two folders are one show when they share
+  an id, or when neither has one and name and year both agree. Two disjoint id sets
+  is an absolute stop: the pair is reported as a conflict and nothing under either is
+  staged. Same name with nothing to decide on is reported unresolved — the fix is the
+  tag, not a guess.
+* **A folder-level verdict destroys content.** 10 of the 25 shows that looked
+  duplicated held COMPLEMENTARY halves: `Game of Thrones Complete` carries no
+  S01 and no S08, `Modern Family S05` is the only copy of season 5, `Loiter
+  Squad Uncensored` is the only copy of season 1. Judged per folder, 20 shows
+  and some 340 GB of unique episodes would have been staged; judged per episode
+  the true figure was a sixth of that.
+* **A bare number is never an episode.** `256491.mp4` may be an episode id, a
+  poster, or a season, so a name that states no `SxxEyy` / `1x04` marker is
+  unkeyed: never staged, and counted in the report. A media server's own episode
+  table does know those files — pass it with `--witness` and its rows key
+  whatever the server has matched.
+* **The audit re-reads the disk.** Sizes and the survivor's existence are
+  checked again when the plan is written, because the census is a snapshot: a
+  file that grew, vanished, or landed in the hold in between must fail the plan,
+  not surprise the executor.
+* **A sidecar that shares an episode's stem follows it.** Subtitles and `.nfo`
+  left at the old coordinates are ghost bait for `media-view.sh reap`.
+* A multi-episode file (`S01E01-02`) is dominated only when every episode in its
+  range is held elsewhere — and it competes for the survivor seat on each of
+  them, because the byte law compares files. A pack that states its range in
+  the name is a name both media servers parse the same way, so the pack winning
+  against two single-episode rips is the law applied, not a misread.
+"""
+import argparse
+import collections
+import importlib.util
+import json
+import os
+import re
+import sys
+
+ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
+_spec = importlib.util.spec_from_file_location("eng", ENGINE)
+eng = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(eng)
+
+# The executor's definition of a media file, plus the two the title engine omits:
+# an audit that disagrees with media-apply.py is a plan that dies at its
+# preflight instead of at the census.
+VIDEO = tuple(sorted(set(eng.VIDEO_SUFFIX) | {".ts", ".wmv"}))
+TAG_RE = re.compile(r"\[\s*tvdbid[\s._-]*(\d+)\s*\]", re.I)
+YEAR_RE = re.compile(r"[(\[]\s*((?:19|20)\d\d)\s*[)\]]")
+SEASON_EP = re.compile(r"(?i)\bs(?:eason)?[\s._-]*(\d{1,2})[\s._-]*e(?:p(?:isode)?)?"
+                       r"[\s._-]*(\d{1,3})(?:[\s._-]*(?:-|to|\+)[\s._-]*e?"
+                       r"[\s._-]*(\d{1,3}))?")
+NXN = re.compile(r"(?i)(?<!\d)(\d{1,2})x(\d{1,3})(?!\d)")
+
+
+def parse_args():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--library", default="/mnt/entropy/TVShows",
+                    help="the TV library to census (never written)")
+    ap.add_argument("--hold", required=True,
+                    help="dated trash the plan stages into, e.g. "
+                         "/mnt/entropy/.AGENTTRASH/20261102-tv-duplicates")
+    ap.add_argument("--out", default=None,
+                    help="plan TSV for scripts/media-apply.py (default: "
+                         "tv-dupes-plan.tsv beside --library)")
+    ap.add_argument("--witness", default=None,
+                    help="TSV of path<TAB>season<TAB>episode out of a media "
+                         "server's episode table; keys files whose names carry"
+                         " no marker")
+    ap.add_argument("--meta", default=None,
+                    help="the merged metadata view (on the host "
+                         "/var/lib/media-meta/view): tvshow.nfo lives in the"
+                         " upper layer, so this is where an untagged folder's"
+                         " tvdb id is readable")
+    ap.add_argument("--ids", default=None,
+                    help="TSV of folder<TAB>tvdbid, one line per folder, for"
+                         " identity a lookup settled and the folder name does"
+                         " not state")
+    ap.add_argument("--json", default=None,
+                    help="also dump the analysis as JSON")
+    return ap.parse_args()
+
+
+ARGS = parse_args()
+
+
+def episode_keys(name):
+    """Every (season, episode) an episode name states.
+
+    `S01E03`, `Season 1 E3`, `1x04`, and a range `S01E01-02`, which expands so a
+    two-in-one file is only droppable when both halves are held elsewhere. A
+    bare number states nothing and yields nothing."""
+    keys = set()
+    for m in SEASON_EP.finditer(name):
+        s, first = int(m.group(1)), int(m.group(2))
+        keys.add((s, first))
+        if m.group(3):
+            keys.update((s, e) for e in range(first + 1, int(m.group(3)) + 1))
+    keys.update((int(m.group(1)), int(m.group(2))) for m in NXN.finditer(name))
+    return keys
+
+
+def folder_identity(folder):
+    """(fold_sep name, year, tvdb ids) for a top-level show folder.
+
+    Tag and year come in both orders — `Name (2008) [tvdbid-84859]` and `Name
+    [tvdbid-84859] (2008)` are both in the library — so neither is assumed to
+    sit at the end, and both are cut out of the name folders compare on."""
+    ids = frozenset(int(x) for x in TAG_RE.findall(folder))
+    year = None
+    for _ in range(3):
+        y = YEAR_RE.search(folder)
+        if not y:
+            break
+        year = year or y.group(1)
+        folder = TAG_RE.sub(" ", f"{folder[:y.start()]} {folder[y.end():]}")
+    return eng.fold_sep(TAG_RE.sub(" ", folder)), year, ids
+
+
+def nfo_id(path):
+    """The tvdb id a `tvshow.nfo` states, or None.
+
+    Three shapes are in the library: Jellyfin's `<uniqueid type="tvdb">`, Kodi's
+    `<tvdbid>`, and a packager's `<id>` + `<id_type>` pair; some carry the id
+    only in the title line as `[tvdbid-N]`."""
+    try:
+        with open(path, encoding="utf-8", errors="replace") as fh:
+            text = fh.read(16384)
+    except OSError:
+        return None
+    for rx in (r"<tvdbid\s*>\s*(\d+)\s*<",
+               r"<uniqueid[^>]*type\s*=\s*[\"']?tvdb[^>]*>\s*(\d+)\s*<",
+               r"<id\s*>\s*(\d+)\s*</id>\s*<id_type>\s*(?:tvdb|thedotdb)",
+               r"\[\s*tvdbid[\s._-]*(\d+)\s*\]"):
+        m = re.search(rx, text, re.I)
+        if m:
+            return int(m.group(1))
+    return None
+
+
+def collect_ids(library, meta, ids_file):
+    """top-level folder -> {tvdb ids} from the nfo layer and an operator map.
+
+    `meta` is the merged metadata view: Jellyfin's sidecars live in the upper
+    layer, so a folder's `tvshow.nfo` is not on the raw array. Reading the id
+    from there is what makes an untagged packager folder identifiable at all —
+    without it the pair stays unresolved and nothing under it is staged."""
+    out = collections.defaultdict(set)
+    if meta:
+        try:
+            tops = sorted(os.listdir(library))
+        except OSError:
+            tops = []
+        for folder in tops:
+            i = nfo_id(os.path.join(meta, folder, "tvshow.nfo"))
+            if i:
+                out[folder].add(i)
+    if ids_file:
+        with open(ids_file, encoding="utf-8", errors="replace") as fh:
+            for line in fh:
+                cols = line.rstrip("\n").split("\t")
+                if len(cols) < 2 or not cols[1].strip().isdigit():
+                    continue
+                out[cols[0]].add(int(cols[1].strip()))
+    return out
+
+
+def witness_rows(path):
+    """absolute path -> {(season, episode)} from a media server's episode table."""
+    out = collections.defaultdict(set)
+    if not path:
+        return out
+    with open(path, encoding="utf-8", errors="replace") as fh:
+        for line in fh:
+            cols = line.rstrip("\n").split("\t")
+            if len(cols) < 3 or not cols[0].startswith("/"):
+                continue
+            try:
+                out[os.path.abspath(cols[0])].add((int(cols[1]), int(cols[2])))
+            except ValueError:
+                continue
+    return out
+
+
+def sidecars(path):
+    """The files beside PATH carrying its exact stem — subtitles, nfo, artwork."""
+    folder, stem = os.path.split(path)
+    stem = os.path.splitext(stem)[0]
+    out = []
+    try:
+        entries = sorted(os.listdir(folder))
+    except OSError:
+        return out
+    for e in entries:
+        p = os.path.join(folder, e)
+        if p == path or os.path.islink(p) or not os.path.isfile(p):
+            continue
+        if os.path.splitext(e)[0] == stem:
+            out.append(p)
+    return out
+
+
+def scan(library, hold, witness):
+    """Every video file under LIBRARY: relative path, size, episode keys.
+
+    A file keys from the witness when it has a row (a server matched it), and
+    from its own name otherwise; with neither it is unkeyed."""
+    files = []
+    for dirpath, dirnames, filenames in os.walk(library):
+        dirnames[:] = sorted(d for d in dirnames
+                             if not d.startswith(".") and d != "@eaDir"
+                             and os.path.abspath(os.path.join(dirpath, d)) != hold)
+        for f in sorted(filenames):
+            p = os.path.join(dirpath, f)
+            if os.path.islink(p) or not f.lower().endswith(VIDEO):
+                continue
+            try:
+                size = os.path.getsize(p)
+            except OSError:
+                continue
+            files.append({"path": os.path.abspath(p),
+                          "rel": os.path.relpath(os.path.abspath(p), library),
+                          "size": size,
+                          "keys": set(witness.get(os.path.abspath(p))
+                                      or episode_keys(os.path.splitext(f)[0]))})
+    return files
+
+
+def show_groups(folders, extra_ids):
+    """(show groups, conflicts, unresolved, disagreements) over top-level folders.
+
+    A folder's id is its `[tvdbid-N]` tag; the nfo layer and `--ids` fill a folder
+    only when it carries no tag. The rule, in order of trust: an id in common makes
+    one show; two disjoint id sets are two shows, reported as a conflict and never
+    merged; no id on either side merges only on name AND year (the 1999 miniseries
+    and the 2015 series are not one show); one side ided and the other bare is
+    unresolved, because the fix is the tag.
+
+    The derived layers never overrule a tag. Measured 2026-11-02: Jellyfin's
+    provider table carried a 2026 special's tvdb id on the folder tagged
+    `[tvdbid-73838]`, and unioning the two made one show of the 2000 series and the
+    special — whose bigger files then dominate the original's pilot. A contradiction
+    is reported instead, because the tag is the operator's word and the provider
+    table is a guess that can be wrong.
+    """
+    names = {f: folder_identity(f) for f in folders}
+    tags = {f: set(names[f][2]) for f in folders}
+    derived = {f: set(extra_ids.get(f, ())) for f in folders}
+    disagree = {f: (sorted(tags[f]), sorted(derived[f] - tags[f]))
+                for f in folders if tags[f] and derived[f] - tags[f]}
+    ids = {f: (tags[f] or derived[f]) for f in folders}
+    parent = {f: f for f in folders}
+
+    def find(f):
+        while parent[f] != f:
+            parent[f] = parent[parent[f]]
+            f = parent[f]
+        return f
+
+    conflicts, unresolved = [], []
+    order = sorted(folders)
+    for i, a in enumerate(order):
+        for b in order[i + 1:]:
+            na, ya, _ia = names[a]
+            nb, yb, _ib = names[b]
+            if na != nb:
+                continue
+            ia, ib = ids[a], ids[b]
+            if ia & ib:
+                parent[find(b)] = find(a)
+            elif ia and ib:
+                conflicts.append((a, b, f"tvdb {sorted(ia)} vs {sorted(ib)}"))
+            elif not ia and not ib and ya and ya == yb:
+                parent[find(b)] = find(a)
+            elif not ia and not ib:
+                conflicts.append((a, b,
+                                  f"same name, years {ya or '?'} vs {yb or '?'}"))
+            else:
+                unresolved.append((a, b, "one folder carries a tvdb id,"
+                                         " the other none"))
+    groups = collections.defaultdict(list)
+    for f in order:
+        groups[find(f)].append(f)
+    return sorted(groups.values()), conflicts, unresolved, disagree
+
+
+def rank(rec):
+    """Survivor order: biggest, then shortest path, then lexicographic.
+
+    A strict total order per key means a key's winner is never itself dominated,
+    so no chain of staging can cycle and exactly one copy of a byte-identical
+    pair is staged."""
+    return (-rec["size"], len(rec["path"]), rec["path"])
+
+
+def main():
+    library = os.path.abspath(ARGS.library)
+    hold = os.path.abspath(ARGS.hold)
+    out = ARGS.out or os.path.join(os.path.dirname(library), "tv-dupes-plan.tsv")
+    if not os.path.isdir(library):
+        sys.exit(f"refused: {library} is not a directory")
+    if hold == library or hold.startswith(library + os.sep):
+        sys.exit(f"refused: --hold {hold} sits inside the library; a hold the"
+                 " census walks is a plan that eats its own output")
+
+    files = scan(library, hold, witness_rows(ARGS.witness))
+    folders = sorted({r["rel"].split(os.sep)[0] for r in files})
+    ids = collect_ids(library, ARGS.meta, ARGS.ids)
+    groups, conflicts, unresolved, disagree = show_groups(folders, ids)
+
+    by_show = collections.defaultdict(list)
+    for members in groups:
+        for r in files:
+            if r["rel"].split(os.sep)[0] in members:
+                by_show[members[0]].append(r)
+
+    rows, audits, report, shows_json = [], [], [], []
+    for show in sorted(by_show):
+        recs = by_show[show]
+        holders = collections.defaultdict(list)
+        for r in recs:
+            for k in r["keys"]:
+                holders[k].append(r)
+        winner = {k: min(v, key=rank) for k, v in holders.items()}
+        dominated = [r for r in recs if r["keys"]
+                     and all(winner[k] is not r for k in r["keys"])]
+        if not dominated:
+            continue
+        dup_keys = sorted({k for r in dominated for k in r["keys"]})
+        sole = sum(1 for v in holders.values() if len(v) == 1)
+        staged_folders = {os.path.dirname(r["rel"]) for r in dominated}
+        for r in dominated:
+            for k in r["keys"]:
+                t = winner[k]
+                try:
+                    now_src = os.path.getsize(r["path"])
+                    now_twin = os.path.getsize(t["path"])
+                except OSError:
+                    audits.append(f"gone since the census: {r['rel']}"
+                                  f" or its survivor {t['rel']}")
+                    break
+                if now_twin < now_src:
+                    audits.append(f"survivor smaller on disk: {r['rel']}"
+                                  f" {now_src} > {t['rel']} {now_twin}")
+        for r in dominated:
+            dst = os.path.join(hold, r["rel"])
+            rows.append((r["path"], dst))
+            for side in sidecars(r["path"]):
+                rows.append((side, os.path.join(hold, os.path.relpath(side, library))))
+        shows_json.append({"show": show, "files": len(dominated),
+                           "bytes": sum(r["size"] for r in dominated),
+                           "episodes": len(dup_keys),
+                           "sole-episode files left": sole,
+                           "folders": sorted(staged_folders)})
+        n_ep, n_folder = len(dup_keys), len(staged_folders)
+        report.append(
+            f"{show}: stage {len(dominated)} files"
+            f" ({sum(r['size'] for r in dominated) / 2**30:.1f} GiB) — {n_ep}"
+            f" episode{n_ep != 1 and 's' or ''} held twice or more; {sole}"
+            f" episode{sole != 1 and 's' or ''} exist in one copy only, so"
+            f" {n_folder} folder{n_folder != 1 and 's' or ''} keep their files")
+
+    rows = sorted(set(rows))
+    dests = set()
+    for src, dst in rows:
+        if dst in dests:
+            audits.append(f"destination twice: {dst}")
+        dests.add(dst)
+        if os.path.exists(dst):
+            audits.append(f"hold target occupied: {dst}")
+        if not os.path.isfile(src):
+            audits.append(f"source missing: {src}")
+
+    unkeyed = collections.Counter(os.path.dirname(r["rel"]) for r in files
+                                  if not r["keys"])
+    if ARGS.json:
+        with open(ARGS.json, "w") as fh:
+            json.dump({"shows": shows_json,
+                       "conflicts": [list(c) for c in conflicts],
+                       "unresolved": [list(u) for u in unresolved],
+                       "disagreement": {f: [t, d] for f, (t, d) in disagree.items()},
+                       "unkeyed": dict(unkeyed)}, fh, indent=1)
+
+    for line in report:
+        print(line)
+    for a, b, why in conflicts:
+        print(f"CONFLICT (never merged): {a} | {b} — {why}")
+    for a, b, why in unresolved:
+        print(f"UNRESOLVED (tag it, then re-run): {a} | {b} — {why}")
+    for f, (tagged, extra) in sorted(disagree.items()):
+        print(f"DISAGREE (the tag wins): {f} — tag tvdb {tagged},"
+              f" derived tvdb {extra}")
+    if unkeyed:
+        print(f"unkeyed, never staged: {sum(unkeyed.values())} files in"
+              f" {len(unkeyed)} folders name no SxxEyy and have no --witness row;"
+              " most: " + ", ".join(f"{d} ({n})" for d, n in unkeyed.most_common(5)))
+
+    # The gate runs before the plan is written: a plan file that exists is a plan
+    # somebody will point the executor at, so a failed audit leaves no artifact.
+    if audits:
+        print(f"AUDIT FAILED ({len(audits)})", file=sys.stderr)
+        for a in audits[:10]:
+            print(f"  {a}", file=sys.stderr)
+        print(f"refused: no plan written to {out}", file=sys.stderr)
+        return 1
+
+    mkdirs = sorted({os.path.dirname(dst) for _s, dst in rows},
+                    key=lambda p: (p.count(os.sep), p))
+    with open(out, "w") as fh:
+        fh.write("action\tsrc\tdst\tkind\n")
+        for d in mkdirs:
+            fh.write(f"mkdir\t\t{d}\tdir\n")
+        for src, dst in rows:
+            fh.write(f"move\t{src}\t{dst}\ttvdup\n")
+
+    held = sum(os.path.getsize(s) for s, _d in rows if os.path.isfile(s))
+    print(f"plan: {len(rows)} moves ({held / 2**30:.1f} GiB) into"
+          f" {len(mkdirs)} hold folders -> {out}")
+    print("audit: every staged file has a survivor in the same show holding each of"
+          " its episodes at >= its size on disk, one destination each")
+    if rows:
+        root = os.path.dirname(library)
+        print(f"next: python3 scripts/media-apply.py apply --plan {out} --root"
+              f" {root} --journal {os.path.join(root, 'tv-dupes.jsonl')}"
+              " --metadata-view /var/lib/media-meta/view   (dry run; add --apply)")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/media-verify.py

@@ -0,0 +1,221 @@
+#!/usr/bin/env python3
+"""Prove a relayout landed, using the media server's own tables.
+
+A scan answers `204` whether or not it found anything, and a rename that the
+scanner has not followed looks exactly like a rename that never happened from
+the outside. The only honest proof is the run's journal replayed against the
+database the server actually serves from: every `move` row must have a row at
+its destination, and NO row may still name its source — a source row is a scan
+that has not caught up, and it is the one signature of a library pointing at
+files that no longer exist.
+
+Two doors, both read-only, both without a lock while the server runs (there is
+no `sqlite3` binary on these hosts). Open `mode=ro` PLAIN: both servers keep
+their libraries in WAL mode, and `immutable=1` — which claims the file never
+changes — makes sqlite skip the write-ahead log entirely, so every row the
+server has committed but not yet checkpointed reads as MISSING, and a read
+beside a running scanner can even fail as "database disk image is malformed".
+`immutable=1` is only the fallback, for a database whose `-shm` cannot be read.
+
+  jellyfin  `BaseItems.Path`. The container binds the share at `/data`, so the
+            database holds `/data/<library>/…` where the journal holds
+            `/mnt/entropy/<library>/…`. Translate, or every row reads as missing.
+  plex      `media_parts.file`, joined up to `metadata_items`. Plex is bound at
+            the host path, so its rows match the journal verbatim. Filter
+            `deleted_at is null` on BOTH tables — a deleted scan leaves its parts
+            behind, and one of those rows is not proof anything landed.
+
+Neither server holds a row for everything the journal renames: a `.log`, `.cue`
+or cover image has no row to land in, so its absence is not a fault. The report
+therefore breaks the missing rows down by extension, and `--ignore-ext` names the
+extensions this server does not index, which are then counted separately and kept
+out of the verdict. What is left is split by what the filesystem says, because
+the database alone cannot: a file still at its `src` is `REAL` (this pass's own
+damage), a file at neither path is `gone` (a later pass or the server renamed it),
+and a file sitting at its `dst` with no row naming it is simply something the
+server does not index. A row that still names `src` is none of these: it is the
+server's own stale pointer — a phantom — and it costs a re-scan, never a restore.
+
+Exit codes: 0 every move landed; 1 something is stale or missing (the details are
+printed); 2 the journal or the database is not usable at all.
+"""
+
+import argparse
+import json
+import os
+import sqlite3
+import sys
+from collections import Counter
+
+JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
+PLEX_DB = ("/var/lib/plexmediaserver/Library/Application Support/Plex Media Server"
+           "/Plug-in Support/Databases/com.plexapp.plugins.library.db")
+
+# Per kind: the table whose columns are required, those columns, and the query
+# that returns every path the server believes it has.
+KINDS = {
+    "jellyfin": {
+        "table": "BaseItems",
+        "columns": {"Path"},
+        "sql": "select Path from BaseItems",
+        "default_db": JF_DB,
+    },
+    "plex": {
+        "table": "media_parts",
+        "columns": {"file", "media_item_id", "deleted_at"},
+        "sql": ("select p.file from media_parts p"
+                " join media_items m on m.id = p.media_item_id"
+                " join metadata_items mi on mi.id = m.metadata_item_id"
+                " where p.deleted_at is null and mi.deleted_at is null"),
+        "default_db": PLEX_DB,
+    },
+}
+
+def die(msg):
+    print(f"media-verify: {msg}", file=sys.stderr)
+    sys.exit(2)
+
+
+def read_moves(path):
+    """The (src, dst) pairs of a journal, in the order the run performed them."""
+    moves = []
+    try:
+        fh = open(path, encoding="utf-8")
+    except OSError as e:
+        die(f"cannot read the journal {path}: {e.strerror}")
+    with fh:
+        for n, line in enumerate(fh, 1):
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                rec = json.loads(line)
+            except json.JSONDecodeError:
+                die(f"{path} line {n} is not a journal record")
+            if rec.get("op") == "move":
+                if not rec.get("src") or not rec.get("dst"):
+                    die(f"{path} line {n} is a move without both src and dst")
+                moves.append((rec["src"], rec["dst"]))
+    return moves
+
+
+def server_paths(db, kind):
+    """Every path the server holds, or a refusal that names what is missing."""
+    spec = KINDS[kind]
+    con = None
+    err = None
+    # Plain ro reads the WAL the server commits through; immutable is the last
+    # resort, because it cannot see a row the server has not checkpointed.
+    for opt in ("mode=ro", "mode=ro&immutable=1"):
+        try:
+            con = sqlite3.connect(f"file:{db}?{opt}", uri=True)
+            break
+        except sqlite3.Error as e:
+            err = e
+    if con is None:
+        die(f"cannot open {db}: {err}")
+    with con:
+        have = {r[1] for r in con.execute(f"pragma table_info({spec['table']})")}
+        if not spec["columns"] <= have:
+            die(f"{db} is not a {kind} library database ({spec['table']} has no "
+                f"{','.join(sorted(spec['columns'] - have))})")
+        try:
+            return {r[0] for r in con.execute(spec["sql"]) if r[0]}
+        except sqlite3.Error as e:
+            die(f"{db} refused the {kind} query: {e}")
+
+
+def to_host(path, container, share):
+    """The database's path in the journal's terms; identity when it is not a
+    container-side path."""
+    if not container:
+        return path
+    prefix = container.rstrip("/") + "/"
+    if path.startswith(prefix):
+        return share.rstrip("/") + "/" + path[len(prefix):]
+    return path
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--journal", required=True,
+                    help="the run's journal (scripts/media-apply.py writes it)")
+    ap.add_argument("--db-kind", choices=sorted(KINDS), required=True)
+    ap.add_argument("--db", default=None, help="the server's database on this host")
+    ap.add_argument("--share", default="/mnt/entropy",
+                    help="the share every journal path lives under")
+    ap.add_argument("--container", default=None,
+                    help="what the share is called inside the container; empty"
+                         " compares verbatim (default /data for jellyfin, nothing"
+                         " for plex)")
+    ap.add_argument("--show", type=int, default=20,
+                    help="how many offenders to name before saying 'and N more'")
+    ap.add_argument("--ignore-ext", default="",
+                    help="comma-separated extensions this server does not index,"
+                         " kept out of the verdict (e.g. log,cue,nfo,srt)")
+    args = ap.parse_args()
+
+    db = args.db or KINDS[args.db_kind]["default_db"]
+    container = ("/data" if args.db_kind == "jellyfin" else "") \
+        if args.container is None else args.container
+    ignore = set()
+    for ext in args.ignore_ext.split(","):
+        ext = ext.strip().lower()
+        if ext:
+            ignore.add(ext if ext.startswith(".") else "." + ext)
+
+    moves = read_moves(args.journal)
+    stored = {to_host(p, container, args.share) for p in server_paths(db, args.db_kind)}
+
+    landed, stale, missing, ignored = 0, [], [], 0
+    for src, dst in moves:
+        if os.path.splitext(dst)[1].lower() in ignore:
+            ignored += 1
+            continue
+        at_dst, at_src = dst in stored, src in stored
+        if at_dst and not at_src:
+            landed += 1
+            continue
+        if at_src:
+            stale.append(src)
+        if not at_dst:
+            missing.append((src, dst))
+
+    # The database cannot tell a lost file from one it never indexes: both read
+    # as "no row at the destination". Stat both paths, so the report separates
+    # the classes the runbook names — `REAL` is the only one that is this pass's
+    # own damage, and `gone` is a later pass or the server's own rename.
+    on_disk, real, gone = [], [], []
+    for src, dst in missing:
+        if os.path.exists(dst):
+            on_disk.append(dst)
+        elif os.path.exists(src):
+            real.append(src)
+        else:
+            gone.append(dst)
+
+    print(f"{args.db_kind}: {len(moves)} moves checked")
+    print(f"landed: {landed}")
+    print(f"still at src: {len(stale)}")
+    print(f"missing from the DB: {len(missing)}")
+    print(f"  on disk at dst, no row names it: {len(on_disk)}")
+    print(f"  REAL — file still at src: {len(real)}")
+    print(f"  gone — at neither path: {len(gone)}")
+    if ignored:
+        print(f"not indexed (extension ignored): {ignored}")
+    if missing:
+        exts = Counter(os.path.splitext(d)[1].lower() or "(no extension)"
+                       for _, d in missing)
+        print("missing by extension: "
+              + ", ".join(f"{e} {n}" for e, n in sorted(exts.items())))
+    for label, items in (("real", real), ("gone", gone), ("missing", on_disk),
+                         ("stale", stale)):
+        for path in items[:args.show]:
+            print(f"  {label}: {path}")
+        if len(items) > args.show:
+            print(f"  {label}: … and {len(items) - args.show} more")
+    return 1 if (stale or missing) else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/media-view.sh

@@ -0,0 +1,258 @@
+#!/usr/bin/env bash
+# The SSD-backed metadata view that lets Jellyfin and Plex share one metadata set.
+#
+# Both servers read a Kodi-format sidecar (movie.nfo / tvshow.nfo / episode.nfo)
+# ONLY when it sits next to the media file. The media lives on the RAID6 array at
+# /mnt/entropy — five spinning disks, deliberately mounted read-only for every
+# container, and slow to seek for a thousand tiny XML reads. So neither media
+# server binds the array any more: they bind an OverlayFS merge of it with a
+# writable layer on the NVMe.
+#
+#   lowerdir  /mnt/entropy                RAID6 (ROTA=1) — source of truth, never written
+#   upperdir  /var/lib/media-meta/upper   NVMe — every sidecar and image byte
+#   workdir   /var/lib/media-meta/work    NVMe — kernel bookkeeping
+#   Where     /var/lib/media-meta/view    the merged tree jellyfin and plex bind
+#
+# Everything the view needs — layers, bookkeeping, trash, the merged door — lives
+# under the one directory /var/lib/media-meta, so nothing new appears under /mnt.
+#
+# Consequences worth knowing:
+#   * A sidecar write is a few KB onto the SSD; the media bytes never move and the
+#     array is still never written by anything automated.
+#   * Deleting through the view writes a whiteout into the upper layer instead of
+#     deleting from the array. Nothing should delete through the view: deletions
+#     go to .AGENTTRASH on the RAW path, and `reap` clears the resulting ghosts.
+#   * inotify events on the lower layer do not cross the merge, so a file that
+#     arrives on the raw array is invisible to a file-system watcher in the view:
+#     trigger a library refresh (see the Media Relayout page).
+#
+# Subcommands
+#   plan   [--upper-base D] [--lower D]  print the resolved shape, touch nothing
+#   verify                               prove the live mount IS that shape
+#   install [--upper-base D]             create the layers, install+enable+start the unit
+#   reap   [--apply] [--upper-base D] [--lower D]
+#                                       list upper-layer entries whose media is gone
+#   status                               one line per fact a human wants
+#
+# install must run as root (it writes /etc/systemd/system and mounts); the others
+# are read-only and safe unprivileged.
+set -euo pipefail
+
+LOWER=/mnt/entropy
+UPPER_BASE=/var/lib/media-meta
+VIEW=$UPPER_BASE/view
+# systemd names a mount unit after its ESCAPED path ('/' -> '-', a literal '-'
+# -> '\x2d'), so the unit on the host is var-lib-media\x2dmeta-view.mount. The
+# repo copy keeps a readable name; install writes it under the escaped one.
+UNIT='var-lib-media\x2dmeta-view.mount'
+UNIT_DIR=/etc/systemd/system
+# The rootless containers write as this host uid (the deploy user). The upper
+# layer has to accept those writes, so it is owned by them.
+MEDIA_UID="${MEDIA_VIEW_UID:-1000}"
+MEDIA_GID="${MEDIA_VIEW_GID:-1000}"
+
+REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+UNIT_SRC="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
+
+die() {
+    printf 'media-view: %s\n' "$*" >&2
+    exit 1
+}
+
+usage() {
+    printf 'usage: %s plan|ensure-base|verify|install|reap|status [options]\n' "$0" >&2
+    exit 2
+}
+
+cmd="${1:-}"
+[ -n "$cmd" ] || usage
+shift || true
+
+upper_base="$UPPER_BASE"
+lower="$LOWER"
+apply=0
+while [ $# -gt 0 ]; do
+    case "$1" in
+        --upper-base) [ $# -ge 2 ] || usage; upper_base="$2"; shift 2 ;;
+        --lower)      [ $# -ge 2 ] || usage; lower="$2"; shift 2 ;;
+        --apply)      apply=1; shift ;;
+        *) usage ;;
+    esac
+done
+
+upper="$upper_base/upper"
+work="$upper_base/work"
+
+# The device backing a path, and whether the kernel calls it rotational.
+rotational_of() {
+    local src rota
+    src="$(findmnt -no SOURCE --target "$1" 2>/dev/null || true)"
+    [ -n "$src" ] || die "cannot find a block device behind $1"
+    rota="$(lsblk -no ROTA "$src" 2>/dev/null | head -1 || true)"
+    [ -n "$rota" ] || die "cannot read the rotational flag of $src"
+    printf '%s %s\n' "$src" "$rota"
+}
+
+require_upper_base() {
+    [ -d "$upper_base" ] \
+        || die "refused: upper base $upper_base does not exist (put it on the SSD first)"
+}
+
+# The SSD placement of the metadata layer is a human choice, so `install` refuses
+# to invent a base directory: a mistyped --upper-base must not quietly create a
+# tree on whatever filesystem the caller landed on. The deployer owns the
+# documented location, so it calls this first — measured 2026-10-31, when a
+# fresh-host `install` died with "put it on the SSD first" and nothing in git
+# would have created it.
+do_ensure_base() {
+    if [ -d "$upper_base" ]; then
+        printf 'base: %s exists\n' "$upper_base"
+        return 0
+    fi
+    install -d -m 0755 "$upper_base" 2>/dev/null \
+        || die "refused: cannot create $upper_base — it lives outside the checkout, run this as root"
+    printf 'base: created %s\n' "$upper_base"
+}
+
+do_plan() {
+    printf 'lowerdir=%s\nupperdir=%s\nworkdir=%s\nmountpoint=%s\nunit=%s/%s\n' \
+        "$lower" "$upper" "$work" "$VIEW" "$UNIT_DIR" "$UNIT"
+}
+
+do_verify() {
+    local fail=0 fst opts src rota
+    if ! mountpoint -q "$VIEW"; then
+        printf 'refused: %s is not a mountpoint — a plain directory here is a\n' "$VIEW" >&2
+        printf '  library that scans itself empty, not a metadata view.\n' >&2
+        return 1
+    fi
+    fst="$(findmnt -no FSTYPE --mountpoint "$VIEW" 2>/dev/null || true)"
+    if [ "$fst" != overlay ]; then
+        printf 'refused: %s is mounted %s, not overlay\n' "$VIEW" "${fst:-nothing}" >&2
+        fail=1
+    fi
+    opts="$(findmnt -no OPTIONS --mountpoint "$VIEW" 2>/dev/null || true)"
+    local want
+    for want in "lowerdir=$LOWER" "upperdir=$UPPER_BASE/upper" "workdir=$UPPER_BASE/work"; do
+        case ",$opts," in
+            *",${want},"*) ;;
+            *) printf 'refused: %s options lack %s (got: %s)\n' "$VIEW" "$want" "$opts" >&2
+               fail=1 ;;
+        esac
+    done
+    read -r src rota <<<"$(rotational_of "$UPPER_BASE")"
+    if [ "$rota" != 0 ]; then
+        printf 'refused: the upper layer sits on %s which is rotational — metadata\n' "$src" >&2
+        printf '  belongs on the SSD, not on the array.\n' >&2
+        fail=1
+    fi
+    if [ ! -w "$UPPER_BASE/upper" ]; then
+        printf 'refused: %s is not writable\n' "$upper" >&2
+        fail=1
+    fi
+    [ "$fail" -eq 0 ] || return 1
+    printf 'view ok: %s <- overlay(%s,%s)\n' "$VIEW" "$LOWER" "$UPPER_BASE/upper"
+}
+
+do_install() {
+    [ "$(id -u)" -eq 0 ] || die "install must run as root (writes $UNIT_DIR and mounts)"
+    require_upper_base
+    mountpoint -q "$LOWER" \
+        || die "refused: $LOWER is not mounted; overlaying an empty directory would present an empty library"
+    local src rota
+    read -r src rota <<<"$(rotational_of "$upper_base")"
+    [ "$rota" = 0 ] || die "refused: $upper_base is on $src, a rotational device — the metadata layer must be on the SSD"
+    [ -f "$UNIT_SRC" ] || die "refused: $UNIT_SRC is missing from $REPO_ROOT"
+    # systemd will not create a mount point; a missing directory fails the start.
+    install -d -m 0755 -o "$MEDIA_UID" -g "$MEDIA_GID" "$upper" "$work" "$VIEW"
+    install -d -m 0700 "$upper_base/.AGENTTRASH"
+    install -m 0644 "$UNIT_SRC" "$UNIT_DIR/$UNIT"
+    systemctl daemon-reload
+    systemctl enable "$UNIT" >/dev/null 2>&1 || true
+    systemctl start "$UNIT"
+    do_verify
+}
+
+# An upper-layer DIRECTORY with no counterpart below is metadata for a media
+# folder that has been moved or renamed on the RAW path — a ghost the media
+# servers would still browse. Files are never candidates on their own: a sidecar
+# (movie.nfo, an .srt) exists ONLY above, by design, and is exactly what the view
+# is for. A copy-up whose lower file was deleted by hand is the one case this
+# leaves in place; the raw path is where deletions belong, so it stays visible.
+do_reap() {
+    [ -d "$upper" ] || die "refused: $upper does not exist (no upper layer at $upper_base)"
+    stamp="$upper_base/.AGENTTRASH/$(date -u +%Y%m%d-%H%M%S)"
+    count=0
+    ghosts=0
+    reap_walk ""
+    if [ "$count" -eq 0 ] && [ "$ghosts" -eq 0 ]; then
+        printf 'reap: no ghosts under %s\n' "$upper"
+        return 0
+    fi
+    if [ "$ghosts" -gt 0 ]; then
+        printf 'reap: %d whiteout%s above — someone deleted THROUGH the view.\n' \
+            "$ghosts" "$([ "$ghosts" -eq 1 ] || printf s)" >&2
+        printf '  Leaving them: removing a whiteout un-hides the file it hides.\n' >&2
+        printf '  Deletions belong in .AGENTTRASH on the raw path.\n' >&2
+    fi
+    if [ "$count" -eq 0 ]; then return 0; fi
+    if [ "$apply" -eq 1 ]; then
+        printf 'reap: moved %d entr%s to %s\n' "$count" \
+            "$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
+    else
+        printf 'reap: %d ghost entr%s above; re-run with --apply to move them to %s\n' \
+            "$count" "$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
+    fi
+}
+
+# Descend only where the lower layer still has the directory: a directory that is
+# missing below is itself the ghost, and its whole subtree goes with it.
+reap_walk() {
+    local rel="$1" entries abs r name
+    entries="$(find "$upper${rel:+/$rel}" -mindepth 1 -maxdepth 1 -print 2>/dev/null || true)"
+    [ -n "$entries" ] || return 0
+    while IFS= read -r abs; do
+        r="${abs#"$upper"/}"
+        name="${abs##*/}"
+        if [ -c "$abs" ] || printf '%s' "$name" | grep -q '^\.wh\.'; then
+            ghosts=$((ghosts + 1))
+            continue
+        fi
+        if [ -e "$lower/$r" ]; then
+            [ -d "$abs" ] && reap_walk "$r"
+            continue
+        fi
+        [ -d "$abs" ] || continue
+        count=$((count + 1))
+        printf '%s\n' "$r"
+        if [ "$apply" -eq 1 ]; then
+            mkdir -p "$stamp/$(dirname "$r")"
+            mv "$abs" "$stamp/$r"
+        fi
+    done <<<"$entries"
+}
+
+do_status() {
+    if mountpoint -q "$VIEW"; then
+        printf 'view: %s (%s)\n' "$VIEW" "$(findmnt -no OPTIONS --mountpoint "$VIEW")"
+    else
+        printf 'view: %s NOT MOUNTED\n' "$VIEW"
+    fi
+    if [ -d "$upper" ]; then
+        printf 'upper: %s %s files %s\n' "$upper" \
+            "$(find "$upper" -type f 2>/dev/null | wc -l)" "$(du -sh "$upper" 2>/dev/null | cut -f1)"
+    else
+        printf 'upper: %s missing\n' "$upper"
+    fi
+    systemctl is-active "$UNIT" 2>/dev/null | sed 's/^/unit: /' || printf 'unit: %s unknown\n' "$UNIT"
+}
+
+case "$cmd" in
+    plan)    do_plan ;;
+    verify)  do_verify ;;
+    ensure-base) do_ensure_base ;;
+    install) do_install ;;
+    reap)    do_reap ;;
+    status)  do_status ;;
+    *)       usage ;;
+esac

scripts/test-changed.sh

@@ -94,6 +94,22 @@ for path in "${CHANGED[@]}"; do
         dankserver/.env.example|dankclaw/.env.example)
             add_test tests/env_consistency.bats
             ;;
+        # The SSD-backed metadata view: the installer, the host mount unit, and the
+        # two media units that bind the view rather than the raw array.
+        scripts/media-view.sh|dankserver/host/*)
+            add_test tests/media_overlay.bats
+            add_test tests/dankserver_quadlet.bats
+            add_test tests/deployment_scoping.bats
+            ;;
+        dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
+            add_test tests/media_overlay.bats
+            ;;
+        # The Audiobookshelf bootstrapper (account + library + scan over the API)
+        # and the deployer hook that runs it after the health gate.
+        scripts/audiobookshelf-init.sh|tests/audiobookshelf_init.bats)
+            add_test tests/audiobookshelf_init.bats
+            add_test tests/deployment_flow.bats
+            ;;
         shared/homepage/*)
             add_test tests/homepage_caddy.bats
             add_test tests/homepage_config.bats
@@ -324,7 +340,7 @@ for path in "${CHANGED[@]}"; do
         # Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
         # copyparty, plex, tautulli): the seam checks boot-persistence +
         # v1-generator contract; deploy-flow seams guard the deployer routing.
-        dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
+        dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*|dankserver/audiobookshelf/*)
             add_test tests/dankserver_quadlet.bats
             add_test tests/deployment_flow.bats
             add_test tests/deployment_scoping.bats

scripts/wait-for-mount.sh

@@ -0,0 +1,74 @@
+#!/usr/bin/env bash
+# wait-for-mount.sh — refuse a container start until what it binds is mounted.
+#
+# Why this exists (live 2026-10-31, Dankserver): a rootless quadlet cannot order
+# itself against a SYSTEM mount unit. `RequiresMountsFor=/mnt/entropy` in a user
+# unit resolves to nothing and is inert, so when the array's mount finished a
+# second after podman resolved the bind, Jellyfin and Plex started against the
+# pre-mount directory. Their library root looked EMPTY and Jellyfin began
+# deleting metadata by the thousand (`Removing item, Type: "Season"`). A blind
+# bind is not a cosmetic fault; it is a metadata-deleting machine.
+#
+# A `ConditionPathIsMountPoint=` is NOT the fix, measured on the same host: an
+# unmet condition logs "skipped, unmet condition check", NRestarts=0, and no
+# start attempt ever follows — the unit stays dead until a human starts it.
+# That is fail-dead. This script WAITS (the mount usually lands within seconds)
+# and only then refuses, so `Restart=always` carries the unit across the race.
+#
+# usage: wait-for-mount.sh [-t SECONDS] MOUNTPOINT [MOUNTPOINT...]
+#   exit 0  every path is a mountpoint
+#   exit 1  a path is still not a mountpoint after the window (named on stderr)
+#   exit 2  usage error
+set -euo pipefail
+
+usage() { printf 'usage: %s [-t SECONDS] MOUNTPOINT [MOUNTPOINT...]\n' "$0"; }
+
+window=90
+while getopts ":t:h" opt; do
+    case "$opt" in
+        t) window=$OPTARG ;;
+        h) usage; exit 0 ;;
+        *) usage >&2; exit 2 ;;
+    esac
+done
+shift $((OPTIND - 1))
+
+[ "$#" -ge 1 ] || { usage >&2; exit 2; }
+case "$window" in
+    '' | *[!0-9]*) printf 'refused: -t wants whole seconds, got %s\n' "$window" >&2; exit 2 ;;
+esac
+
+is_mount() { findmnt -M "$1" -rno TARGET >/dev/null 2>&1; }
+
+report() {
+    local why=$1; shift
+    printf '%s:' "$why"
+    for p in "$@"; do printf ' %s' "$p"; done
+    printf '\n'
+}
+
+deadline=$(( $(date +%s) + window ))
+warned=0
+while :; do
+    missing=()
+    for path in "$@"; do
+        is_mount "$path" || missing+=("$path")
+    done
+    [ "${#missing[@]}" -eq 0 ] && break
+
+    if [ "$(date +%s)" -ge "$deadline" ]; then
+        report "refused: still not mounted after ${window}s" "${missing[@]}" >&2
+        printf 'refusing to start: a container bind-mounted here would see the empty\n' >&2
+        printf 'directory underneath and conclude the library is empty — which is how a\n' >&2
+        printf 'scan begins deleting metadata it cannot find.\n' >&2
+        exit 1
+    fi
+    if [ "$warned" -eq 0 ]; then
+        report "waiting for mount(s) this unit depends on" "${missing[@]}"
+        warned=1
+    fi
+    sleep 1
+done
+
+[ "$warned" -eq 0 ] || printf 'mounted; proceeding.\n'
+exit 0

serverAppConfigs-docs/Agent/Issue Tracker.md

@@ -48,6 +48,14 @@ Use the `Authorization: token` header for everything.
 - **Read an issue**: `curl -s -K "$cfg" $API/issues/<number>` (fields: `number`, `title`, `body`, `labels[].name`, `state`)
 - **List issues**: `curl -s -K "$cfg" "$API/issues?state=open&limit=50"` — add `&labels=<id>` to filter
 - **Comment**: `curl -s -K "$cfg" -X POST -d "$(jq -n --rawfile b "$BODY" '{body:$b}')" $API/issues/<number>/comments`
+- **Comment on a PR**: the conversation thread is `POST $API/issues/<PR-number>/comments`
+  (201, and the answer's `html_url` points at `pulls/<n>`; Gitea's issue index is
+  unified, so the PR's number is the same number). `$API/pulls/<n>/comments` is the
+  **review**-comment store and answers `405` to a plain body.
+- **A body built by `jq --rawfile b /dev/stdin` inside `$(…)` is empty**: the command
+  substitution does not inherit the heredoc, so Gitea receives `{}` and answers 422 —
+  and `jq -r '.id'` on the error prints nothing, which reads as success. Write the
+  body to a file and pass `--rawfile` that file.
 - **Apply labels**: `POST $API/issues/<number>/labels` with `{"labels":[<label-ID numbers>]}` — Gitea wants numeric label **IDs**, not names; resolve them from `GET $API/labels` (see [[Triage Labels]] for which labels exist)
 - **Remove a label**: `DELETE $API/issues/<number>/labels/<label-id>`
 - **Close**: `PATCH $API/issues/<number>` with `{"state":"closed"}`

serverAppConfigs-docs/Architecture/Deployment Flow.md

@@ -68,8 +68,8 @@ service on the other host.
 | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
 
 The shared chat template maps to the inference service on the selected target.
-The `frank-npu` endpoint is retired () and has no deploy handler
-an unrelated large-model deployment never touches it.
+The `frank-npu` endpoint is retired and has no deploy handler, so an unrelated
+large-model deployment never touches it.
 
 ## Caddy safety
 

serverAppConfigs-docs/Architecture/E2E Suite.md

@@ -100,3 +100,12 @@ aborts with `DEPLOY INCOMPLETE: e2e gate FAILED` when it does not pass
    hand-off list (`HANDOFFS`) instead of a single host. The guard contract is
    untouched: challenge cleared, no authentik `uidField`, status < 500, and every
    host NOT in the list still has to land on its own vhost.
+7. **The runner's resolver is part of the suite.** A 2026-10-31 deploy of one
+   Dankserver quadlet failed the gate with 21 reds spread over SSO, homepage RBAC
+   and every vhost crawl — precisely the specs that navigate to an absolute
+   `*.franksdank.cloud` name, while every spec on the relative `baseURL` stayed
+   green. The error was `net::ERR_NAME_NOT_RESOLVED`: the workstation had lost its
+   wired link, and on the fallback its resolver was no longer pinned to Pi-hole.
+   Nothing on the fleet was wrong — the same suite re-run minutes later passed
+   37/37. Before a mass red gets blamed on the guard chain, run
+   `resolvectl query grafana.franksdank.cloud` on the box that runs the gate.

serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md

@@ -167,6 +167,43 @@ Gate: `tests/quadlet_image_pins.bats`. Where upstream publishes no version
 tags (copyparty-ac stops at 1.19.15), pin `:latest@sha256:` and name the
 version the config label carries in the comment beside it.
 
+## 4b. Resource control needs host permission (measured 2026-10-31, Dankserver)
+
+`IOWeight=` in a rootless quadlet does nothing on its own, and nothing says so.
+The system manager delegates `cpu memory pids` to `user@.service` and **not** `io`;
+systemd drops a resource line it cannot honour **without a word** — the generator
+still emits `IOWeight=10`, the container still starts and answers its health check,
+and the cgroup simply has no `io.weight` file. Same family as
+`RequiresMountsFor=` (inert in the user manager) and `ConditionPathIsMountPoint=`
+(fail-dead): the line is accepted, the unit is happy, the setting is dead.
+
+Two system drop-ins make it real, both placed by `ensure_user_io_delegate` in
+`scripts/deploy-remote.sh` (repo copies under `dankserver/host/systemd/`):
+
+| File | Effect |
+|---|---|
+| `user.slice.d/io-accounting.conf` → `[Slice] IOAccounting=yes` | puts `io` into `user.slice/cgroup.subtree_control`; lands immediately, no restart |
+| `user@1000.service.d/io-delegate.conf` → `[Service] DelegateControllers=cpu memory pids io` | the user manager can only request controllers it was delegated; read when it starts, so it lands on **the next reboot** (restarting it kills every rootless container on the host) |
+
+Read the chain in this order — each line is the failure mode of the one after it:
+
+```bash
+cat /sys/fs/cgroup/user.slice/cgroup.subtree_control                    # wants: io
+systemctl show user@1000.service -p DelegateControllers                 # wants: io
+cat /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/\<unit\>.service/libpod-payload-*/io.weight
+```
+
+`io` missing from the first → the accounting drop-in is absent. Missing from the
+second → the host has not rebooted since the delegation landed (the deployer prints
+`NOTE: io is not delegated …` for exactly this). No file at the third → that unit
+carries no weight.
+
+Two limits that survive the fix: a container's weight covers **everything** in its
+cgroup, so a library scan and a playback stream share one weight; and the comparison
+is made *inside* `user.slice`, so a rootless unit's weight cannot push it below a
+root unit (`frank-archon`). Shaping that pairing is the weight of `user.slice`
+itself — a fleet decision, not a unit line.
+
 ## 5. Rootless Quadlet log retrieval
 
 Step 0 — establish ownership: `systemctl --user status <name>.service`
@@ -195,8 +232,9 @@ deploy.sh, never a live env poke.
    check every namespace (`sudo podman ps -a`) and native units
    (`systemctl list-units`) before declaring a service absent. Compare
    compose mount sources against the real filesystem — compose files name
-   phantom paths (e.g. `/mnt/media/*` vs the real LUKS array at
-   `/mnt/entropy`). Fix compose through git first, deploy second.
+   phantom paths (the legacy `dankserver/compose.yaml` still names `/opt/jellyfin/*`
+   for a service whose roots moved to `/home/dank/Containers/jellyfin`). Fix
+   compose through git first, deploy second.
 2. **Pre-create bind-mount sources** (`sudo mkdir -p` + `chown 1000:1000`)
    or creation fails `statfs …: no such file or directory`.
 3. **Staged bring-up:** infra first (`up -d mariadb redis`), poll

serverAppConfigs-docs/Architecture/Testing Policy.md

@@ -23,7 +23,7 @@ This repository tests its configuration at defined seams. The tests run with `ba
 | Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
 | Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
 | E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] |
-| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
+| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{audiobookshelf_init,anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
 
 ## Focused pre-commit testing
 

serverAppConfigs-docs/Dankclaw/Frank TTS.md

@@ -222,5 +222,5 @@ harness bundle and measured the day of the flip:
 | Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
 | First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
 | The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
-| A consumer rejects `wilwheaton` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
+| A consumer rejects `wilwheaton` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed. In OpenWebUI the `browser-kokoro` choice is that user's own `user.settings` row (`ui.audio.tts.engine`) and the SPA re-serves it on every load, so flipping the dropdown does not stick; see [[OpenWebUI]] |
 ---

serverAppConfigs-docs/Dankclaw/OpenWebUI.md

@@ -21,7 +21,7 @@ https://github.com/open-webui/open-webui
 | Item | Value |
 |------|-------|
 | Image | `ghcr.io/open-webui/open-webui:main` (verified from the live quadlet unit) |
-| Deployed build | `0.11.4` — `GET http://127.0.0.1:3005/api/version` → `{"version":"0.11.4"}` (2026-10-18). `:main` is the rolling image, not a stable pin, so the running build lags upstream: releases `0.12.0` (2026-10-10) through `0.12.3` (2026-10-16) exist and are not deployed (https://github.com/open-webui/open-webui/releases). The 2026-09-21 `0.11.4` security/access-control advisory is what pinned this era of the page. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
+| Deployed build | `0.12.5` — `GET http://127.0.0.1:3005/api/version` → `{"version":"0.12.5"}` (re-measured 2026-11-04). The repo pin `@sha256:0b399a8b…` (`dankclaw/open-webui/quadlet/open-webui.container`, bumped 2026-10-30) is what makes that build reproducible; `:main` is a rolling tag, so re-measure `/api/version` instead of quoting a build from this page. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
 | Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
 | Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
 | Backends | The picker's content is `webui.db` state, not repo state: as recorded on 2026-10-07 it exposes **one** model, `frank-zealot`, with `frank-dragoon`/`frank-archon` configured but disabled — *unverified 2026-10-18* (the DB is the authority and is not readable from the repo). What the repo does fix is the container's own OpenAI target: `OPENAI_API_BASE_URL=http://192.168.0.143:7770` (`dankclaw/open-webui/quadlet/open-webui.container:32`, same value in the live container env) — see [The LLM backends](#the-llm-backends) |
@@ -109,6 +109,39 @@ The Ollama connection is disabled (`ollama.enable=false`).
 > connection here only hides it from this picker — the endpoint itself keeps
 > serving every other client (the harness, Hermes, API consumers).
 
+## Default model, descriptions, and the admin API door
+
+**The default model is `ui.default_models`** — one row of the config table, a
+comma-separated **string**, not a list. Live value `frank-archon` since
+2026-11-04 (per Frank). The front end resolves the picker as *the browser's own
+stored pick, else `config.default_models.split(",")`*, so the value decides what
+a fresh profile or a cleared-storage session lands on; after a user picks
+something themselves, their pick keeps winning. Verified 2026-11-04 in a clean
+browser: signed in with `localStorage` empty → the pill reads
+`Selected model: frank-archon`.
+
+Two admin-token doors, both driven against loopback `:3005` — service-to-service
+traffic never rides the public vhost:
+
+- **Config:** `POST /api/v1/configs/import` with
+  `{"config":{"ui.default_models":"frank-archon"}}` — the general per-key setter
+  (no `PATCH /configs/{key}` route exists in 0.12.5; verified against
+  `routers/` + `main.py`), and it returns the whole config so the write reads
+  back in the same call.
+- **A model row:** `POST /api/v1/models/model/update` with a full `ModelForm`
+  (`id`, `name`, `meta`, `params`, `is_active`). **`meta` and `params` REPLACE,
+  they do not merge:** read the row first, change one field, send everything
+  back — a partial payload silently drops `meta.capabilities` and the
+  profile/background images.
+
+**An API key is not accepted everywhere.** Routes guarded by `get_human_user`
+(`/api/v1/auths/`, `utils/auth.py`) answer **403 `A human session is required.`**
+for any token whose `typ` is not `session` — that is not a dead key. The two
+mutation doors above take `get_admin_user` / `get_verified_user`, which do accept
+the key. Prove a key on `GET /api/v1/models` with **no trailing slash**:
+`/api/v1/models/` falls through to the SPA and answers `200` + `text/html`, which
+looks like success and parses as nothing.
+
 ## Container introspection (DB and backend python)
 
 - Working dir for imports: `/app/backend`; package name `open_webui`
@@ -163,14 +196,11 @@ account under Profile → API Keys). Anonymous web visitors get the `pending`
 role (`ui.default_user_role=pending`), which cannot chat. Verified
 2026-10-07 from loopback `:3005` and over the public vhost:
 `GET /api/v1/models/list`, `GET /api/v1/models` and
-`POST /api/chat/completions` all return **401** with no key (also with a
-bogus bearer token); only `/api/config`, `/api/version` and `/health`
 `POST /api/chat/completions` all return **401** with no key (also with a bogus bearer
 token); only `/api/config`, `/api/version` and `/health` answer publicly, and none of
 them expose the model list. An external program needs an account + API key — there is
-no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`
-`/api/version` → 0.11.4, `/api/v1/models/list`, `/api/v1/models` and
-`POST /api/chat/completions` → 401; `/api/config` additionally publishes
+no keyless access. Re-measured 2026-11-04 from loopback: `/api/version` → `0.12.5`,
+and `/api/config` publishes
 `features.enable_login_form=false`, `features.enable_signup=false` and
 `oauth.providers.oidc="authentik"` — the SSO-only posture, visible from outside.
 
@@ -219,18 +249,26 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
   into the existing local account with the same email — the pre-SSO admin
   (frank8465@protonmail.com) keeps its data. `ENABLE_OAUTH_SIGNUP=true` lets
   other `openwebui_users` create accounts on first login.
-- **First login mints role `pending`.** `ui.default_user_role=pending` (config
-  DB, verified 2026-10-09), so a freshly-provisioned `openwebui_users` account
-  completes SSO and then cannot chat (picker empty) until its role is flipped:
-  `UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` —
-  run with host `python3` over ssh against
+- **First login mints the role from the claim, not from
+  `ui.default_user_role`.** The quadlet sets `ENABLE_OAUTH_ROLE_MANAGEMENT=true`
+  and `OAUTH_ROLES_CLAIM` defaults to `roles`, so the `roles: ["user"]` claim
+  decides the role. Measured 2026-11-04: a freshly-provisioned
+  `openwebui_users` account completed SSO in a clean browser and the row
+  appeared as `role='user'` — chat answered and Read Aloud synthesized, with no
+  manual step. `ui.default_user_role=pending` (config DB) is the fallback for an
+  account the claim does *not* describe, and 0.12.5's `get_user_role` **refuses
+  the login with 403** when role management is on and no roles claim is
+  readable — so "SSO succeeded but I cannot chat" points at a missing
+  `openwebui_users` group/entitlement, not at a row waiting to be flipped. Flip
+  only a row that is actually stuck:
+  `UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` — run
+  with host `python3` over ssh against
   `/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
-  Dankclaw; table is `user`, singular; no `active` column). No per-user model
-  grant is needed: the picker model `frank-zealot` carries the global
-  `access_grant` row `(model, frank-zealot, user, *, read)`, so any
-  `role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
-  not exist until the account signs in (2026-10-09: five provisioned accounts
-  had no row).
+  Dankclaw; table is `user`, singular; no `active` column). The row does not
+  exist until the account signs in, so census it after a first-login sighting
+  (2026-10-09: five provisioned accounts had no row). No per-user model grant is
+  needed: the picker models carry the global `access_grant` row
+  `(model, <id>, user, *, read)`, so any `role='user'` account sees them.
 - **IdP reachability:** the container is on its own bridge (10.89.3.0/24) and
   cannot reach `authentik-server:9000` by name. It reaches the IdP over the
   public URL `https://auth.franksdank.cloud` — Caddy publishes `0.0.0.0:443`
@@ -305,28 +343,58 @@ port is the rootless quadlet `frank-tts` (this repo:
 400 listing the voices that exist, never a silent fallback.
 
 **The 29-voice `af_* … bm_fable` list is NOT frank-tts.** That enum belongs
-to **Kokoro.js running in the browser**: the deployed front end (0.11.4)
-ships an in-page Kokoro implementation (chunk carrying a frozen voice table
-`{af_heart:{name:"Heart",…}}` and `_validate_voice`, model
-`onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
-setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
-`browser-kokoro` is chosen the page synthesizes locally and rejects any
-voice outside its fixed pack — so `wilwheaton` fails with
-`Voice: "wilwheaton" not found. Should be one of: af_heart, …` **without a
-single request reaching the server**. Personal audio settings are
-browser: no `user.settings` row and no container env carries an audio
-override, yet the toast still appears ⇒ the choice lives in that browser's
-own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
-which hands synthesis to the admin config (frank-tts); picking `af_heart`
-instead keeps Kokoro, and Kokoro cannot clone — fixed pack only.
+to **Kokoro.js running in the browser**: the deployed front end ships an
+in-page Kokoro implementation (frozen voice table plus `_validate_voice`,
+model `onnx-community/Kokoro-82M-v1.0-ONNX`, and a personal *Kokoro.js dtype*
+setting). Its engine select takes `""` (Default) or `browser-kokoro`; with
+`browser-kokoro` the page synthesizes locally and rejects any voice outside
+its fixed pack, so `wilwheaton` fails with `Voice: "wilwheaton" not found.
+Should be one of: af_heart, …` **without a single request leaving the
+browser**. Kokoro cannot clone — fixed pack only.
 
-Verified 2026-10-18 from inside the container with the exact OpenWebUI body
-(`{"model":"frank-tts","input":…,"voice":"narrator1",
-"response_format":"mp3","speed":1.0}`) → **200, `audio/mpeg`, 7,629 B**,
-while `podman logs open-webui` recorded **zero** `audio/speech` requests for
-the failing click — that contrast is the instrument: a server-path failure
-leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
-neither.
+**The switch is a DB row, not browser storage.** Personal audio settings are
+the `settings` JSON column of table `user`, and in 0.12.x they nest under
+`ui`: `ui.audio.tts.engine`, `.voice`, `.defaultVoice`. In 0.11.x the same
+keys sat at the top level (`audio.tts.*`), and the 2026-10-18 probe listed
+only top-level keys, found no audio key, and concluded the choice lived in
+that browser's own storage — a probe-depth bug. The row carried the override
+the whole way; the 2026-10-30 pin bump to 0.12.5 moved the keys, not the bug.
+The SPA reads that row at load (`GET /api/v1/users/user/settings`) and posts
+the **whole** blob back whenever any preference changes
+(`POST …/settings/update`, body `{...settings}`). So the dropdown can fix it —
+but a second tab still holding the old copy writes `browser-kokoro` straight
+back, which is the loop that makes the toast return after it was "fixed". The
+user row is the authority here; the admin `config` table was never the problem
+(its `audio.tts.engine` has been `openai` throughout). Repair
+`settings.ui.audio.tts` with the image's stdlib `sqlite3` (`mode=ro`; quote
+identifiers, `user` and `group` are reserved words — there is no `sqlite3` CLI
+in the image or on the host), then reload every tab of that account.
+
+**Repair the engine *and* the voice.** With engine `""` the front end still
+sends its stored voice in the request body, so a Kokoro-era `voice`/
+`defaultVoice` survives the engine fix as frank-tts's own rejection —
+measured 2026-11-04: `voice: "alloy"` → **400 `unknown voice 'alloy';
+available voices: dr.kern, garrickhagon, jeffhays, vaati, wilwheaton`**. Set
+`ui.audio.tts` to `{"engine": "", "voice": "wilwheaton", "defaultVoice":
+"wilwheaton"}` and reload. Engine `""` means "the admin config decides", and
+the handler fills `voice` from `audio.tts.voice` only when the body omits it.
+
+**Two tells settle the case in one line:**
+
+- a session that fetched `/wasm/ort-wasm-simd-threaded.jsep.wasm` loaded
+  onnxruntime, so Kokoro.js is running in that browser — the server path
+  never asks for that file;
+- a server-path failure leaves a uvicorn request line for
+  `/api/v1/audio/speech` **and** a frank-tts log line; a `browser-kokoro`
+  failure leaves neither. uvicorn logs every request, so silence is evidence.
+
+Verified 2026-11-04: the handler's own outbound shape
+(`{"model":"frank-tts","input":…,"voice":"wilwheaton"}` with
+`Authorization: Bearer none`, the configured `audio.tts.openai.api_key`) →
+**200 `audio/wav`** — a body without `response_format` comes back WAV and
+either MIME is browser-playable, so OpenWebUI accepts it. The full route as a
+real user over the vhost, `POST /api/v1/audio/speech` → **200 `audio/mpeg`**,
+with frank-tts logging `speech voice=wilwheaton`.
 
 ## Retired instances
 
@@ -396,6 +464,21 @@ neither.
   app process env; the `.webui_secret_key` file happens to equal it — use it
   explicitly, `podman exec -e`).
 
+- **2026-11-04 — decoration rows drifted again; default model set to
+  `frank-archon`.** Two payload swaps had gone by without anyone touching the
+  picker: `frank-dragoon`'s row still advertised halogen `0.16.2`, and
+  `frank-zealot`'s still said `build 10777` / `3×262,144` / *MTP off* — live is
+  `0.18.20`, `build 12118`, `3 × 200,704`, MTP armed. The prose is cosmetic; the
+  `params.context.length` values had **swapped**, and that one bites:
+  `frank-zealot` advertised 262,144 while the endpoint refuses anything past its
+  200,704 slot, so a long zealot chat rides the endpoint's 400 instead of
+  compacting, and `frank-dragoon` advertised 200,704 against a 262,144 ceiling,
+  giving up 61k tokens of window for nothing. Both rows re-keyed through the API
+  door in [Default model, descriptions, and the admin API door](#default-model-descriptions-and-the-admin-api-door),
+  descriptions rewritten from the env files, and `ui.default_models` set to
+  `frank-archon`. **Law: a model swap is not finished until that endpoint's
+  `model` row is re-keyed as well — no deployer step touches `webui.db`.**
+
 ## See Also
 
 - [[Dankclaw Stack]]

serverAppConfigs-docs/Dankserver/Audiobookshelf.md

@@ -0,0 +1,368 @@
+---
+type: reference
+description: "Audiobookshelf on Dankserver: quadlet shape, array bind, SSD state, API bootstrap, authentik SSO, public vhost, laws."
+tags:
+  - dankserver
+  - audiobooks
+  - quadlet
+timestamp: 2026-11-03
+---
+
+# Audiobookshelf
+
+Audiobookshelf (ABS) serves the [[Entropy Volume]] audiobook library to browsers
+and phones: it scans `/mnt/entropy/Audiobooks`, keeps its own SQLite library with
+per-listener playback positions, and streams the audio. It is **not** a media
+server — it shares nothing with [[Plex]] or [[Jellyfin]] and owns no files on the
+array.
+
+**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user manager) —
+**Doors:** LAN `http://192.168.0.235:8000/audiobookshelf` (plain HTTP, nothing in front) and public `https://audiobooks.franksdank.cloud/audiobookshelf/` — Caddy straight to the container, **exempt from the guard chain** because the app must answer non-browser clients ([[Caddy]] *Unguard a vhost*). The public name is the address every client carries: Pi-hole answers it with `192.168.0.143` on the LAN, Porkbun answers it with the WAN address outside, and Caddy serves a real certificate either way.
+
+## Shape
+
+| Item | Value |
+|------|-------|
+| Unit file | `dankserver/audiobookshelf/quadlet/audiobookshelf.container` (the authority for every number here) |
+| Image | `ghcr.io/advplyr/audiobookshelf` pinned by **amd64 manifest digest** — see Laws |
+| Port | `Environment=PORT=8000` + `PublishPort=8000:8000` — web UI and API on one port |
+| Base path | `Environment=ROUTER_BASE_PATH=/audiobookshelf` — every URL below carries this prefix, and the shipped SPA cannot boot without it (Laws) |
+| Network | `pasta` (standalone; publishes nothing to other containers) |
+| Audio bind | `/mnt/entropy/Audiobooks:/audiobooks:ro` — read-only, by design |
+| State | `/home/dank/Containers/audiobookshelf/config` → `/config`, `…/metadata` → `/metadata` (both NVMe) |
+| Mount gate | `ExecStartPre=…/wait-for-mount.sh /mnt/entropy` |
+| Health | `wget` against `/healthcheck` inside the container, 30 s / 3 retries |
+| I/O share | `IOWeight=10` — a scan loses the array to every other tenant |
+| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
+| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
+| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
+| Single sign-on | authentik provider `audiobookshelf`; the local `frank` account stays enabled as break-glass — see *Single sign-on* |
+| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl` |
+
+## Bootstrap
+
+`scripts/audiobookshelf-init.sh` — run by the deployer after the health gate, so a
+deployed ABS is a *usable* ABS: `/healthcheck` answers 200 on a server with no
+user, no library and no scan, and nothing else in the deploy notices the
+difference. The script is idempotent; re-running it costs one incremental scan.
+
+| Step | Door | Printed |
+|------|------|---------|
+| credential | `~/.config/secrets/audiobookshelf` (mode 600, generated once, host-side) | `credential: created …` |
+| account | `POST /init` with `{"newRoot":{"username","password"}}` | `account: created (frank)` / `already registered` |
+| session | `POST /login` → bearer token | `token: obtained (root)` |
+| library | `POST /api/libraries`, folder `/audiobooks`, provider `audiobook` | `library: created Audiobooks (<id>)` / `exists` |
+| scan | `POST /api/libraries/<id>/scan` | `scan: accepted` |
+| proof | `GET /api/libraries/<id>/items?limit=1` | `items: <total>` |
+
+Sign in at `http://192.168.0.235:8000/audiobookshelf` as `frank`; the password is
+`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on
+Dankserver. Change it in **Settings → Users → frank → Password**; the secret file
+then holds a stale value and only matters again if the `config` bind is ever
+destroyed (a fresh DB makes the script create a new owner from it).
+
+That value is the live password, not a leftover: it bcrypt-matches `users.pash` in the state
+DB (2026-11-03), so `local` does open the door when authentik is down.
+
+A full scan is slow and runs in the background; watch the `items:` total climb,
+never `/healthcheck`. It also reads files that are genuinely broken and logs
+`[AudioFileScanner] Invalid data found when processing input` per file — that is
+the file, not the bind: the same line count grows while the total climbs.
+
+## Phone app
+
+The official **Audiobookshelf** app connects with Server Address
+`https://audiobooks.franksdank.cloud/audiobookshelf` — the base path belongs in the address —
+username `frank`, and the same password as the web UI, or the SSO button. One address works
+at home and away: on the LAN Pi-hole answers that name with `192.168.0.143` and Caddy dials
+the container; off-LAN the same name resolves to the WAN address and takes the same route
+(inbound 443 is forwarded at the router — fetched from a datacenter node 2026-11-03). The raw
+LAN address still answers and is the break-glass door whenever Caddy is down. `/ping`
+(`{"success":true}`) and `/healthcheck` (`OK`) answer at the ROOT whatever `ROUTER_BASE_PATH`
+says, so the app's connectivity probe works before it resolves the base path at all, and
+`download: true` on the account is what makes offline listening work.
+
+**A home device that bypasses Pi-hole cannot reach it.** The G36 does not hairpin: resolving
+the name to the WAN address from inside the LAN times out (measured 2026-11-03). Only a
+client pinned to an external resolver hits this — everything using the router's DNS gets the
+LAN answer.
+
+**The login token is at `user.token`, not top level.** `POST /login` answers
+`{ user, serverSettings, ereaderDevices, Source, userDefaultLibraryId }` and the bearer sits
+inside the user object. A client that reads `response.token` gets the empty string, sends
+`Authorization: Bearer ` with nothing after it, and every `/api/*` call then answers `401`
+as though the password were wrong.
+
+**`POST /api/login` is not a route.** Local login is `POST <base>/login`, and the `/api`
+bearer guard answers anything else with a bare `401 Unauthorized` while logging **nothing** —
+no `[LocalAuth]` line — so a wrong path is indistinguishable from a wrong password unless the
+journal is read too. Same family as the absent `/setup` route named in
+`scripts/audiobookshelf-init.sh`. To settle a suspected stale credential without signing in:
+bcrypt-compare the recorded value against `users.pash` in
+`/home/dank/Containers/audiobookshelf/config/absdatabase.sqlite`, opened read-only
+(`file:<path>?mode=ro`).
+
+## Listeners (accounts)
+
+The door is public now, so the account is the only gate between a listener and the library.
+Two ways in, both per listener:
+
+- **Local account** — **Settings → Users → Add User**: type `user`, tick only the libraries
+  they may hear, and leave `download` off unless they need offline listening. Hand them the
+  server address above and their own password.
+- **Household SSO** — create the authentik account, and that is the whole of it: the first
+  **Sign in with Authentik** auto-registers the ABS account (`authOpenIDAutoRegister`) as a
+  listener, because the claim grants that role to every signer. To shut a listener off,
+  disable or delete their user in Audiobookshelf — no IdP group membership can withhold it.
+
+What auto-registration hands out (read off the state DB, 2026-11-03): `accessAllLibraries:
+true` and `download: true` — every household signer lands on the **whole** library with
+offline downloads already on. Narrow it per account in **Settings → Users → <name>**. The DB
+is the census when no admin token is at hand; `GET /api/users` returns the *limited* shape
+(no `active`, no `librariesAccessible`), so a missing key there is not a denial, and an
+account created by SSO has `pash` NULL with its IdP link at `extraData.authOpenIDSub`.
+
+Nothing rate-limits `POST /audiobookshelf/login` — this Caddy build carries no `rate_limit`
+module and ABS 2.37.2 ships no lockout — so the form is a guessing target exactly like
+`jelly.`, `plex.` and the *arr logins already are. The mitigation is credential hygiene:
+long unique passwords per listener, never a reuse of the root password.
+
+## Single sign-on (authentik)
+
+Both doors take the household identity: the web UI puts **Sign in with Authentik** beside
+the local form, and the phone app offers SSO on its server-address screen. The local
+`frank` account stays enabled on purpose — `authActiveAuthMethods` is `local` **and**
+`openid`, and `local` is how you get in when the IdP is down.
+
+| Piece | Owner |
+|-------|-------|
+| Provider `audiobookshelf`, the admin group, the scope mapping | `scripts/authentik-audiobookshelf-sso.sh` — re-run it after an authentik rebuild, because this state lives in the IdP's database and a blueprint re-import reverts it silently |
+| Client secret | `ABS_OIDC_CLIENT_SECRET` in `~/.config/secrets/audiobookshelf` on Dankserver (mode 600; never in the repo, a URL or an argv) |
+| The ABS half: issuer, the four IdP URLs, claim, auto-register | `scripts/audiobookshelf-init.sh`, armed automatically when that secret exists in the credential file |
+| Role at first SSO sign-in | the `abs_groups` claim: every signer gets `user`; `audiobookshelf_admins` adds `admin`. Nothing in the IdP can deny the listener role — the login itself is the gate (law since 2026-11-03; the `audiobookshelf_users` group was retired, it had zero members and every household account was being refused) |
+| Admin group members | `akadmin` (the operator's own authentik admin account — sign in as it and the claim grants site admin) and `qa-e2e-admin` (the identity the deploy gate signs in as, so the script re-adds it every run; drop it and the gate fails, not a phone) |
+| Guard state of the door | none since 2026-11-03: proxy provider `audiobooks-proxy` and application "Audiobookshelf Guard" were deleted and the embedded outpost un-claimed them, because that chain answers a native client with challenge HTML ([[Caddy]] *Unguard a vhost* for the order). The SSO relying party `audiobookshelf` above is a different object and stays |
+| Redirect URIs on the RP | `<base>/auth/openid/callback` **and** `<base>/auth/openid/mobile-redirect` for **both** origins (LAN address and public name): the app sends the mobile URI belonging to the door it was configured with, and STRICT matching answers an unregistered one with "Redirect URI Error" |
+
+With `authOpenIDAutoRegister` on, a first sign-in creates the account and the claim alone
+decides its type — and the claim always carries `user`, so **the IdP is now the library gate**:
+any authentik account that can reach this application becomes a listener. That is the
+intent (the household signs in with the accounts it already has), and its breadth is
+accepted on purpose: no access policy is bound to the `audiobookshelf` application, and
+the wide auto-registered default (every library, downloads on) is **not a defect to fix** —
+Frank's call, 2026-11-03. Narrowing stays a per-account judgement in **Settings → Users**.
+The one thing still worth watching is authentik's own invitation/signup flow: it must
+remain something only Frank drives, because an account created there is a listener the
+moment it signs in. Revoking one person is a disable/delete of their ABS user; revoking
+everyone is that signup flow.
+
+Both halves — authentik's provider row and ABS's settings — are asserted end to end by the Playwright gate named in *Verification*. `23-` signs in as `qa-e2e-admin` and must land inside the app with site-admin rights; `24-` signs in as the standard identity, which is in no Audiobookshelf group, and must land as a listener. Together they pin the escalation and the default: a mapping that stops granting silently locks the household out, and a mapping that over-grants turns the household into site admins.
+
+## Laws
+
+- **Speak the image's env names, not the README's.** The server reads `PORT`,
+  `CONFIG_PATH`, `METADATA_PATH` and `ROUTER_BASE_PATH`; the `ABS_*` family printed
+  by the server's own README is the npm package's naming and is **inert** in this
+  image. Measured 2026-11-02 on the first deploy: with `ABS_PORT=8000` the server
+  listened on `:80` under base path `/audiobookshelf`, so `systemctl` said
+  `active (running)` while every probe of `:8000` was refused and the deploy gate
+  failed at 180 s. Two seconds with the pinned digest settles it:
+  `podman inspect --format '{{.Config.Env}}' <image>`.
+- **The URL carries `/audiobookshelf`, and that is not a style choice.** The SPA in
+  `/app/client/dist/index.html` is built with `routerBasePath:"/audiobookshelf"`, so it asks
+  for its assets under that prefix whatever the server is mounted at. Mounted at `/`, every
+  script request falls through to the SPA fallback, comes back `text/html`, Chrome refuses to
+  execute it, and the page never leaves "Loading…" — while `/healthcheck`, the API and the
+  phone app stay perfectly healthy, which is how this hid behind an API-only verification
+  (measured 2026-11-02). Prove a UI claim in a browser, never from `/health`.
+- **The OIDC redirect subfolder is a value you send, not a default you inherit.** The 2.17.4
+  upgrade only fills `authOpenIDSubfolderForRedirectURLs` on a database that ALREADY had OIDC
+  enabled, so a DB created before SSO leaves it unset and the server builds
+  `http://host/undefined/auth/openid/callback`. authentik answers the authorize request with
+  "Redirect URI Error" and nothing on the ABS side looks wrong (measured 2026-11-02). The
+  provider's registered URI must equal `<origin><subfolder>/auth/openid/callback`, so each
+  script derives the value from one variable of its own (`ABS_URL`, `ABS_BASE`) and
+  `tests/audiobookshelf_init.bats` pins what reaches the wire.
+- **An ORM-created provider has no grant types.** authentik 2026.8 defaults `grant_types` to
+  `[]`, and `/application/o/authorize/` then refuses the code flow with oauthlib's generic
+  "The request is otherwise malformed" — the wire names nothing, the server log line that names
+  it is `Invalid grant_type for provider`. Worse, the redirect-URI check runs *first*, so a
+  provider carrying both faults shows only "Redirect URI Error" until the URI is fixed, and the
+  second wall appears after the fix ships (measured 2026-11-02: the browser reached authentik,
+  got `code`+`state`, and ABS still bounced to `/login?error=undefined`). The script sets
+  `authorization_code` + `refresh_token`; a blueprint re-import reverts the row silently.
+- **The guarded door has to tell the truth about its scheme.** ABS validates every OIDC
+  `callback` against the origin it believes it is served from — `getRequestOrigin()` is
+  `x-forwarded-proto://Host` (`server/utils/requestUtils.js`, read in the pinned digest
+  2026-11-03) — so behind a TLS edge that reaches the container over plain http it computes
+  `http://audiobooks.franksdank.cloud` and answers its own https callback with a 400 "Invalid callback
+  URL - must be same-origin" before the IdP is ever contacted. The Caddy `:8017` route pins
+  `header_up X-Forwarded-Proto https` for exactly that reason ([[Caddy]] *Guard a new vhost*);
+  the LAN door needs nothing, because there browser and server agree on `http`.
+- **The claim mapping is not the whole `property_mappings` list.** A provider holding only the
+  custom mapping grants a token scoped shorter than the scopes ABS requests
+  (`openid profile email abs_groups`): the code is issued and exchanged normally, and ABS's
+  `/userinfo` call then dies `insufficient_scope (The request requires higher privileges than
+  provided by the access token)` — a fault that reads like an ABS bug and is an IdP row. Attach
+  the built-in OpenID `openid`/`profile`/`email` mappings beside the claim mapping, which is
+  what every working RP on this IdP carries (measured 2026-11-02).
+- **With `authOpenIDMatchExistingBy` unset, an SSO identity never lands on an existing ABS
+  account, and a name collision is a hard denial.** In that mode `findUserFromOpenIdUserInfo`
+  matches on `sub` only, so every first sign-in auto-registers a NEW user; if the IdP username
+  equals an ABS username — the local root `frank` is the one here — the create hits the unique
+  constraint and the callback dies `Unauthorized`. So "create a `frank` user in authentik" does
+  not reach the root account: either set `authOpenIDMatchExistingBy` to `username` (links the
+  IdP account to the existing one, and `setUserGroup` refuses to downgrade a root — root plus an
+  `admin` claim is allowed, root plus `user` is denied) or pick a non-colliding IdP name. Read
+  off `server/models/User.js` and `server/auth/OidcAuthStrategy.js` in the pinned digest
+  (2026-11-02).
+- **The audio bind stays `:ro`.** ABS writes nothing that belongs beside the
+  audio: its library, covers, progress and sidecar metadata all land in
+  `/metadata` on the SSD. A writable bind would put stray writes on the spindle
+  and hand the array to a service with no reason to hold it. If per-book
+  `.opf`/`.nfo` sidecars on disk are ever wanted, the rollback is one line —
+  point the audio bind at the [[Media Metadata View]] the media servers already
+  use — but that view's upper layer exists for those two servers' sidecars, so a
+  third writer needs a reason first ([[Media Metadata View]]).
+- **Do not bind the raw array without the mount gate.** In a rootless unit
+  `ConditionPathIsMountPoint=` is fail-*dead* and `RequiresMountsFor=` is inert,
+  so a missing array does not stop the start: podman invents an empty directory,
+  ABS indexes it, and an empty library becomes recorded truth. `wait-for-mount.sh`
+  is the only gate that works here — same law the media servers obey
+  ([[Quadlet Lifecycle]]).
+- **The `HealthCmd` must use `wget`.** The image is Alpine and ships `wget`, not
+  `curl`; a `curl` health command reads a perfectly healthy server as unhealthy
+  and `Restart=always` turns that into a loop.
+- **The image pin is the amd64 manifest digest, not the index digest.**
+  `podman inspect` reports the platform manifest (`RepoDigests[0]`) for what was
+  pulled, which differs from the manifest-list digest `latest` points at. Pin the
+  one this host runs. `latest` is newer than the newest version tag: upstream
+  stopped publishing version tags after `2.19.0` while `latest` kept moving, so
+  "highest version tag" is stale here — read the index, don't guess from tags.
+- **`IOWeight=10` is inert without the host drop-ins.** `dankserver/host/systemd/`
+  carries `user.slice.d/io-accounting.conf` and the `user@1000` delegate, and the
+  deployer installs both; without them systemd silently drops the weight
+  ([[Quadlet Lifecycle]]).
+- **No `systemctl --user enable`.** Generated Quadlet units cannot be enabled on
+  this host; `WantedBy=default.target` in the unit file is what carries the
+  service across a reboot ([[Quadlet Lifecycle]]).
+- **One door for every client, and it sits outside the guard chain on purpose.**
+  `https://audiobooks.franksdank.cloud/audiobookshelf` is what browsers and apps use;
+  `http://192.168.0.235:8000/audiobookshelf` is the raw LAN socket that same Caddy dials,
+  so it doubles as break-glass when the proxy is down. Behind the chain (until
+  2026-11-03) the vhost answered the phone app's `/ping` and `/socket.io/` handshake with
+  Anubis' challenge HTML — Anubis demands a JavaScript proof and the outpost needs a
+  session cookie, and a native client solves neither — so the app evaluated
+  `'isInit' in e.data` on a non-object, reported "Failed to contact server", and the
+  container's request log never recorded the attempt. Podcast RSS and OPDS readers died
+  the same death. ABS's own auth is the gate, as it is for [[Plex]] and [[Jellyfin]]
+  ([[Caddy]] *Unguard a vhost*).
+- **There is no `/setup` route; the creation door is `POST /init`.** `/status`
+  (no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with
+  `{"newRoot":{"username","password"}}` creates it, `POST /login` returns the
+  bearer every `/api/*` route wants. `POST /setup` answers `Unauthorized` — that
+  is a missing route wearing an auth message. Read off `dist-server/server/Server.js`
+  inside the pinned digest rather than from a forum.
+- **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and
+  `/init` and the scan route use it; a client that parses every success as JSON
+  crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it).
+- **The item count is not a field on the library DTO, and paging never ends
+  itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
+  that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
+  is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is
+  ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while
+  `page=1` shares none of them. A census that pages on `offset` therefore reads
+  `ceil(total / limit) × limit` rows containing only `limit` distinct items — the
+  exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs
+  reported for one library. Page on `page`, count *unique ids*, stop on `total`,
+  and take the number after the pass settles (`total` moves during a scan).
+- **A fresh scan is the only thing that notices new audio.** `inotify` does not
+  cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
+  law — so the in-container watcher sees nothing a host process files onto the
+  array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
+- **One stray media file at a collection root turns the whole collection into one
+  book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`),
+  and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the
+  FIRST directory along its path that already has a group: the first file seen at a
+  root owns every deeper file, and the folders below it never become items. One
+  949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a
+  single item of 5,693 tracks, titled after whichever file supplied metadata.
+  "Hundreds of books are missing" is a layout fault, not a scan fault.
+- **"Media file" means audio OR ebook, so an audio-only census cannot see the file
+  that is hiding the library.** With `audiobooksOnly=false` (the deployed shape)
+  the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio
+  extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A
+  census reporting "that root holds no loose audio" can be correct and still
+  useless — the file doing the shadowing here was a comic archive.
+- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed
+  from disk, so the next scan rebuilds the item from the same stray file (measured:
+  `DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the
+  MOVE: give every stray root file a folder of its own (one file per folder is
+  ABS's own single-book layout; formats of one book share a folder), through the
+  journal-guarded mover, then delete the stale item so the old path is not kept,
+  then scan.
+- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of
+  `POST /libraries/:id/scan` walk the disk, so finding new folders needs neither
+  and the deployer's cheap trigger stays on the plain arm. Force additionally
+  re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600
+  items) and repaired nothing on a library whose residuals were file-level: item
+  count, path set and media-less count identical before and after.
+- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In
+  `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then
+  `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
+  swallowed subtree — `hard=1` there deletes every book beneath it off the array.
+  Assert the item's `path` first, and never pass `hard`.
+- **`Invalid data found when processing input` names the file, not the bind.** The
+  scanner logs it per file and skips that file; a book whose files ALL fail never
+  becomes an item at all, so the disk-vs-DB folder census shows it as a book with
+  no item. That is corrupt media, not a lost book. Never clear a suspect file with
+  a lenient probe: `ffprobe -show_entries format=duration` happily returns three
+  hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the
+  log's file paths instead, and `stat -c %s` first — 0-byte files are the common
+  case, and a truncated download reports `moov atom not found`.
+- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension
+  census counts those folders; ABS deliberately does not — its grouping attaches
+  their files to the ancestor item (measured: 110 such folders under one item
+  carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap.
+- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping
+  predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE`
+  (`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no
+  media. A media-less row beside those lines is the file, not the scanner.
+- **A scan is finished when a NEW scan uuid completes.** Match
+  `Library scan "<uuid>" completed in ...`. `podman logs --since 30m` is a sliding
+  window: the set of visible completion lines changes on its own as old lines age
+  out, so a poller keyed on that hash reads a mid-scan number and calls it done.
+
+## Verification
+
+`./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats`
+(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the `wget`
+health check, the base path the shipped SPA needs), `tests/audiobookshelf_init.bats` (the
+bootstrapper against a stub of the API above: create-once, idempotent re-run, no secret in a
+URL, refusal by URL when the server does not answer, the deployer calling it only after the
+health gate, and the SSO arm sending the mounted base path as the redirect subfolder);
+`tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may reference the metadata view);
+and the live Playwright gates `tests/e2e/specs/23-audiobookshelf-sso.spec.ts` and
+`tests/e2e/specs/24-audiobookshelf-sso-default-listener.spec.ts`, run by `tests/e2e/run.sh` at
+the end of every `deploy.sh`. `23-` (the `admin` project) signs in as `qa-e2e-admin` against
+`http://192.168.0.235:8000/audiobookshelf` and asserts the two things no stub can see: every
+`.js` the app loads arrives as JavaScript (a bundle built for the wrong base path arrives as
+`text/html`, which boots nothing), and `/api/me` answers 200 with `type: "admin"` — true only
+while grant types, the OpenID scope mappings and the group claim all hold together. `24-` (the
+`standard` project) signs in as `qa-e2e`, an account in no Audiobookshelf group, and must come
+back `type: "user"`: the default grant that lets the household in, so a scope-mapping
+regression surfaces as a failed sign-in in the gate instead of as a friend's complaint.
+
+The public door is pinned by the guard audit `tests/e2e/specs/14-guard.spec.ts`, which now
+carries an **exemption** assertion where it used to carry a guarded one: an anonymous raw
+client must get the app's own `{"success":true}` on `/ping`, a machine-format socket
+handshake, and the app's own `401` on `/audiobookshelf/api/libraries` — never challenge
+HTML. `tests/anubis.bats` pins the same shape in the Caddyfile: the vhost never proxies to
+`anubis:8923`, and the post-auth listener carries no `@audiobooks` route.
+
+Five book folders hold media no scanner can read, so they stay out of the library:
+three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom,
+one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty
+Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the
+latter two also exist in readable form under their author folders. Fix by
+re-acquiring the files; the scanner names every one of them in its log.

serverAppConfigs-docs/Dankserver/Entropy Volume.md

@@ -1,51 +1,89 @@
 ---
 type: reference
-description: "The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at"
+description: "Dankserver's /mnt/entropy volume: the LUKS/array chain, the crypttab + fstab boot path, and the containers bound to it"
 tags:
   - dankserver
-timestamp: 2026-08-28
+timestamp: 2026-10-31
 ---
 
 # Entropy Volume
 
-The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at
-`/mnt/entropy`. The command-line helper reuses the host's desktop-integrated
-`udisks2`, `crypttab`, and `fstab` configuration; it does not store or handle
-the passphrase itself.
+`/mnt/entropy` is the ext4 filesystem on the LUKS container holding `/dev/md0`
+(RAID6, seven disks). The media stack's data lives there, so the mount is a boot
+dependency for those containers rather than a convenience.
 
-## Command-line mount
-
-```bash
-/home/dank/serverAppConfigs/scripts/mount-entropy.sh
-```
-
-Run it as the normal desktop user (`dank`), not from a root shell. `udisksctl`
-will securely prompt for the LUKS passphrase. `sudo mount /mnt/entropy` then
-uses the existing `/etc/fstab` entry, preserving its options:
+## Chain
 
 ```text
-nosuid,nodev
+/dev/md0                     <- mdadm (its mdadm.conf ARRAY is what makes md0 exist at boot)
+  -> LUKS UUID=cd86c728-459e-4f3a-bcc2-41ecf84685ec   keyfile: /etc/entropy
+     -> /dev/mapper/entropy  <- systemd-cryptsetup@entropy.service
+        -> ext4 at /mnt/entropy  <- mnt-entropy.mount, from /etc/fstab
 ```
 
-The existing mountpoint is owned by `dank:dankserver` with mode `0755`; the
-helper deliberately does not run `chmod` or `chown`, so it cannot silently
-change those permissions.
+## Files that own it
 
-## Device path
+| Repo file | Installs to | Produces at boot |
+|---|---|---|
+| `dankserver/host/crypttab` | `/etc/crypttab` | `systemd-cryptsetup@entropy.service` → `/dev/mapper/entropy` |
+| `dankserver/host/fstab` | `/etc/fstab` | `mnt-entropy.mount` (mountpoint `dank:dankserver`, `0755`) |
 
-The current storage chain is:
+Deploy a change by copying the file onto the host, then
+`systemctl daemon-reload && update-initramfs -u`. The initramfs step is not
+optional: the cryptsetup hook bakes `/etc/crypttab` into the initrd, and the
+array is unlocked there, before the real root mounts. `scripts/mount-entropy.sh`
+(`udisksctl unlock` + `mount`) stays as the manual path for a volume that was
+never unlocked; on a correct boot it is redundant.
 
-```text
-/dev/md0 -> LUKS UUID cd86c728-459e-4f3a-bcc2-41ecf84685ec
-         -> ext4 label entropy
-         -> /mnt/entropy
-```
+## Law
 
-The helper checks that `/dev/md0` exists and exits successfully if the volume
-is already mounted. It is intentionally specific to this host and volume:
-that avoids accidentally unlocking or mounting a different disk.
+**`crypttab`'s device field takes the LUKS UUID, never the md array UUID.**
+`blkid` prints both on adjacent lines for a RAID-backed container, and the file
+shipped since 2025 carried the array UUID (`61bd6eaf-…`) where the LUKS container
+is `cd86c728-…`. The generator produced a perfectly valid unit pointing at a
+device that never appears, so the mapper came into existence only through a
+desktop `udisks2` session — the volume looked mounted on every login while every
+boot failed to mount it. The matching `fstab` line named the same wrong UUID
+directly instead of the mapper.
 
-## Related
+**A LUKS keyfile is hashed whole, so a trailing newline is a different
+passphrase.** The shipped `/etc/entropy` was 27 bytes — 26 characters plus `\n` —
+and `cryptsetup` rejected it even after the UUID was corrected. Keyfiles stay
+`0600` with no trailing newline; prove it with
+`cryptsetup open --test-passphrase --key-file /etc/entropy UUID=…` *before*
+relying on it for a boot.
 
-- [[Storage Policy]]
-- [[Dankserver Stack]]
+**Prove a mount fix with the artifacts systemd generates, not with a live
+mount.** The regenerated `systemd-cryptsetup@entropy.service` naming the right
+device, `findmnt /mnt/entropy` sourcing `/dev/mapper/entropy`, and a matching
+checksum between the repo copy and the host copy are the evidence. A mount that
+exists because someone logged in proves nothing about the boot path — that
+exactly describes the year this bug survived.
+
+## Consumers
+
+`jellyfin`, `copyparty`, and `plex` bind paths under `/mnt/entropy`; with the
+volume absent they start `active (healthy)` against empty directories, which is
+how a missing 91 TiB volume hides. See [[Dankserver Stack]] for the stack and
+[[Storage Policy]] for what may be written here.
+
+## Write doors
+
+Something renamed movie folders on this array with no journal in any of the
+`/mnt/entropy/*.jsonl` plans, no bash history, no cron, no timer, and no job
+artifact. The census of things that can mutate it:
+
+- **A local process** as `dank` or root. `scripts/media-apply.py` is the only
+  one that journals; anything else (a hand `mv`, another session's script) is
+  silent.
+- **SFTP by a human.** GNOME Files on a workstation mounts the array through
+  `gvfsd-sftp`, which runs `sftp-server` on this host as `dank`. Renames and
+  moves are direct `rename()` calls: no shell, no history, no journal. A folder
+  rename bumps **only that directory's ctime** — its mtime and every child's
+  ctime stay at their old values, so the files look untouched while their paths
+  are gone. Diagnose it with `pgrep -af gvfsd-sftp` on the workstation and
+  `ss -tnp` for the peer.
+- **Copyparty is not a door**: the bind is `ro` at the mount level, so no HTTP
+  or WebDAV write reaches the array regardless of account flags ([[Copyparty]]).
+- **No SMB or NFS server exists here** — the `*arr` stack on FrankBook-Pro
+  cannot reach the array at all, so it cannot rename, import, or rescan it.

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -1,9 +1,9 @@
 ---
 type: reference
 title: Jellyfin
-description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
+description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
 tags: [jellyfin, dankserver, media-server]
-timestamp: 2026-09-03T00:00:00Z
+timestamp: 2026-11-03T11:40:00Z
 ---
 
 # Jellyfin
@@ -18,7 +18,7 @@ clients.
 public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 [[Authentik]] outpost (post-auth route proxies `localhost:8096`).
 
-## Facts (verified from the live unit, 2026-09-03)
+## Facts (the repo quadlet — deployed as the live unit on 2026-10-31)
 
 | Item | Value |
 |------|-------|
@@ -26,7 +26,7 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 | Image | `ghcr.io/jellyfin/jellyfin:latest` |
 | Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
 | Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
-| Media | `Volume=/mnt/entropy:/data:ro` — the whole [[Entropy Volume]] mounted read-only; nothing automated writes to it |
+| Media | `Volume=/var/lib/media-meta/view:/data` — the [[Media Metadata View]] overlay, **writable**, live since the 2026-10-31 cutover: sidecars land on the NVMe and the [[Entropy Volume]] array underneath stays untouched |
 | GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
 | Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
 | Health | `curl -f http://localhost:8096/health` every 30 s |
@@ -36,11 +36,144 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 ```
 /home/dank/Containers/jellyfin/config   -> /config   (JELLYFIN_DATA_DIR)
 /home/dank/Containers/jellyfin/cache    -> /cache    (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
-/mnt/entropy                            -> /data     (read-only media volume)
+/var/lib/media-meta/view                -> /data     (git: the overlay view)
+/mnt/entropy                             -> /data     (the overlay's lower layer; also the rollback bind)
 ```
 
-Library folders point at `/data` paths; because the mount uses the same
-absolute path as on the host, library paths survive container recreation.
+Library folders point at `/data` paths: stable across container recreation and
+identical to the raw array layout below. Which side of that path the container
+sees is the [[Media Metadata View]] overlay, live since the 2026-10-31 cutover:
+`/var/lib/media-meta/view:/data`, read-write.
+
+`Local metadata` — save metadata to media folders — is enabled, and it is the
+reason the view exists: `movie.nfo`, `tvshow.nfo` and `episode.nfo` written once
+on the NVMe upper layer are files [[Plex]] can read too — and since the
+2026-10-31 cutover that is what happens: Jellyfin is the fleet's only metadata
+writer, and every sidecar it saves is an SSD byte the array never sees.
+**Artwork is not one of those sidecars.** No image saver is bound for any
+library, so posters and fanart go to the internal store
+(`config/metadata/library/<xx>/<item-id>/poster.jpg`), which [[Plex]] never
+reads — movie art has never existed beside the media on either layer.
+
+**Library coverage is the first thing to check when a title is missing.** The
+four libraries are `movies → /data/Movies`, `tvshows → /data/TVShows`,
+`music → /data/Music`, `boxsets → /config/data/collections`
+(`GET /Library/VirtualFolders`). The review pile — `/data/NeedsProcessing`, the
+former `media-park`, drained into it 2026-11-02 — is in **none** of them, so
+everything filed there is invisible to Jellyfin by configuration,
+not by fault: a recursive refresh answers 204, burns CPU for minutes and
+indexes nothing, because it never looks there. Check coverage before re-scanning.
+
+**Which read proves an item is indexed: the folder view, never the recursive type
+list.** `GET /Items?IncludeItemTypes=Movie&Recursive=true` answered 2,101 items on
+2026-11-02 while `GET /Items?ParentId=<Movies ItemId>` listed 2,548 children — every
+folder on disk — and the eight folders the recursive read "lost" (three `Gundam 00
+Special Edition`, five `Sailor Moon Memorial`) each answer as `Type: Movie` through
+`SearchTerm` and sit in the folder listing. The two reads are not interchangeable, so
+the recursive list is a wrong instrument for "did the refresh index what I moved": its
+shortfall is grouping, not missing media, and acting on it starts a scan that fixes
+nothing. Verify per item — `ParentId` listing or `SearchTerm` — and use the recursive
+count only as a trend line.
+
+**Failure signature:** `System.IO.IOException: Read-only file system :
+'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
+instead of the view — always a fault on this host since the 2026-10-31 cutover.
+`scripts/media-view.sh status` says which: the quadlet drifted back to the raw
+array, or the mount died.
+
+**`EnableInternetProviders` false turns a library into a silent file index.**
+Every refresh answers `204`, stores nothing and logs nothing — no provider ids,
+no plot, no rating, no art — while the item count reads like a healthy scan. The
+switch lives in `config/root/default/<Library>/options.xml`, Jellyfin 12's name
+for the per-library option file: grepping for the old `library.xml` concludes "no
+per-library options exist", and the database holds none, so the trail dies there.
+Two more traps in the same file: `TypeOptions/ImageFetchers` is the *enabled* set,
+so an empty `<ImageFetchers />` switches art off while `ImageFetcherOrder` stays
+fully populated — order is not enablement — and the populated `<MetadataFetchers>`
+built next to it makes the library look healthy.
+
+**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`, cross-checked
+against each library's `config/root/default/<Library>/options.xml`. The switches are NOT
+top-level DTO keys: the folder carries `Name`, `CollectionType`, `Locations`, `ItemId`,
+`PrimaryImageItemId`, `RefreshStatus` and `LibraryOptions`, and every per-library option
+sits inside `LibraryOptions`. Reading `EnableInternetProviders` off the folder itself
+answers `None` for all four — the camelCase trap one level deeper, and it looks exactly
+like an unconfigured library.
+
+| Library | `CollectionType` | `EnableInternetProviders` | `EnableRealtimeMonitor` |
+|---|---|---|---|
+| `Movies` | movies | true (fixed 2026-11-02) | **false** |
+| `Shows` | tvshows | true (fixed 2026-11-02) | true |
+| `Music` | music | **false** | true |
+| `Collections` | boxsets | **false** | false |
+
+`Music` and `Collections` are silent file indexes in the sense above: a refresh answers
+`204`, stores nothing, logs nothing. Any metadata or artwork work on either one
+starts by flipping the switch, never by firing a scan. `Shows` sat in that set
+until 2026-11-02; the door is `POST /Library/VirtualFolders/LibraryOptions` with
+the library's own `ItemId` (below), and `scripts/jellyfin-providers.sh verify` is
+the one command that says which libraries are still in it.
+
+**The per-library option door on 12.x is `POST /Library/VirtualFolders/LibraryOptions`**
+— `GET` there answers `405`, which is how the route announces itself. The 10.x doors
+are gone: `/Library/VirtualFolders/Update` and `/Library/VirtualFolders/Refresh`
+answer `404` on this build, so every recipe naming them fails outright.
+
+**A library record can be a phantom.** A per-library option capture taken 2026-11-01
+carried seven entries — `Shows2`, `Shows3`, `Music2` among them, each with
+`Locations: []` and a live-looking `ItemId` — while `GET /Library/VirtualFolders` the
+same day listed four. A numeric suffix on a library name is what a deleted and
+recreated library leaves behind. The live set is the entries whose `Locations` are
+non-empty; count those and never the records.
+
+**`RemoteImages` is the call that separates a provider fault from a library
+fault.** `GET /Items/{id}/RemoteImages?Type=Primary` returns the provider's real
+poster list even with every automatic pass switched off — plugin, key and CDN all
+proven by that one response — so a refresh that stores nothing beside it is a
+library-switch problem, never a provider problem. It is also the delivery door:
+`POST /Items/{id}/RemoteImages/Download` saves the chosen image to the internal
+store on a build where the automatic image pass will not run at all — it is how
+the Movies library got its art: 2,515 of 2,564 films carry a primary now, and the
+rest have none in TMDb (shorts like `Astartes`, files whose names match nothing).
+
+**The Download door's exact wire shape:** the list response carries the address in
+`Url` — there is no `ImageUrl` field — and the POST takes camelCase query params,
+`?type=Primary&imageUrl=<Url>`, answering `204`; the stored tag then appears in
+`ImageTags`. PascalCase params, or an `imageUrl` left empty, answer `500`, which
+reads as a server fault and is a field-name fault.
+
+**`RemoteImages` answers empty for an item with no `ProviderIds`** — no provider id,
+nothing to look up. Art coverage is capped by identification, not by the image door:
+the items a campaign returns nothing for are the unidentified ones, and no image
+retry will change that.
+
+**A `Download` pass reports zero only when it is pinned to one provider.** A TMDb-pinned
+campaign over `Shows` classified every item in 20 s, downloaded nothing, and concluded that
+identification was the only lever left. The same door queried with no `Providers=` filter
+answered for 16 seasons that same day — every offer from `TheTVDB`, which the pinned form
+never asks — and all 16 are dressed now. A pinned answer proves one provider is empty; the
+unpinned answer is the provider census, and a TV art loop that pins is simply the wrong tool.
+Post-fill the artless remainder measures 86 items (80 seasons, 6 series): 61 carry no provider
+id at all — the unnamed long tail (`3a BBW - Series (2010-11)`, `History Of Africa With Zeinab
+Badawi`, `(Warhammer TV)`-shaped directories), where the lever is folder names or an identify
+pass — and 25 hold an id and draw no offer from any provider. No image retry moves either
+group, which is the half of the old conclusion that survives.
+
+**Series and season art is fetched; episode stills are not, deliberately.** Every
+episode query is a fresh provider round-trip — nothing caches per episode — measured
+at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a
+sample. Across 42k episodes that is days of provider traffic to dress a fifth of them,
+and clients display the series and season art anyway. If episode stills are ever
+ordered, the loop is the same door and its log makes the pass resumable.
+
+**API shape on 12.x:** `/Items/{id}` needs `UserId` when authenticated by API key — without
+it the answer is `400 Error processing request`, which reads exactly like an authentication
+failure, and it bites hardest as a *verification* read directly after a write that succeeded.
+The `/Items` list route, `?Ids=<id>` included, needs no `UserId`, so verify writes there.
+`MetadataRefreshMode`
+accepts `Default` (and `Realtime`) only; `Full` and `ValidationFailure` are
+rejected outright, and what forces the providers to re-run is
+`ReplaceAllMetadata=true`.
 
 ## Ports & access
 
@@ -55,7 +188,7 @@ Client access paths and the public-exposure policy (auth wall required before
 any public vhost; Plex-style claim tokens do not exist here — Jellyfin uses its
 own users/API keys) follow the fleet [[Authentik]]/[[Anubis]] chain pattern.
 
-## Authentik LDAP authentication (2026-09-25)
+## Authentik LDAP authentication (live since 2026-09-25, re-verified 2026-11-04)
 
 Jellyfin has no native external-auth, so it authenticates users through the
 **LDAP-Auth plugin** (`jellyfin-plugin-ldapauth`) against an [[Authentik]] LDAP
@@ -71,17 +204,35 @@ user to verify the password.
   publishing `389`/`636` on the Dankclaw host for cross-host binds.
 - Service account `svc-jellyfin-ldap` (DN
   `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`).
-- Access groups `jellyfin_users` (search filter
-  `(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`) and
-  `jellyfin_admins` (admin filter `(cn=jellyfin_admins)`). Add a user to
-  `jellyfin_users` to grant them Jellyfin access; `jellyfin_admins` grants
-  the Jellyfin admin role.
+- Access group `jellyfin_users`: the plugin's search filter is
+  `(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`, so a user
+  who is not in that group is **invisible** to the plugin and the login fails
+  as "Wrong username or password". Add members with
+  `Group.objects.get(name="jellyfin_users").users.add(user)` in `ak shell`
+  (directory data — the DB owns it, not this repo).
+  `jellyfin_admins` is **inert as configured** — see the admin-filter law.
 
 **Jellyfin side** (plugin config, set via admin → Plugins → LDAP-Auth):
 - Server `192.168.0.143:389` (plain LDAP over the trusted LAN), Base DN
   `dc=franksdank,dc=cloud`, bind user = the service-account DN, search
   attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`
   `CreateUsersFromLdap=true`, all folders enabled.
+- Admin base `ou=groups,dc=franksdank,dc=cloud`, admin filter
+  `(cn=jellyfin_admins)`, `EnableLdapAdminFilterMemberUid=false`.
+
+**Law — the admin filter grants nothing unless it can return the user's own
+DN.** With `EnableLdapAdminFilterMemberUid=false` the plugin searches
+`LdapAdminBaseDn` with `LdapAdminFilter` and sets the admin permission only when
+a returned entry's DN equals the bound user's DN
+(`LDAP-Auth/LDAPAuthenticationProviderPlugin.cs`). A group-OU base returns the
+*group* entry, whose DN never matches, so **no LDAP user can ever become a
+Jellyfin admin** — measured 2026-11-04: the search answers
+`cn=jellyfin_admins,ou=groups,…` for every login. Working shape if group-granted
+admins are ever wanted: base `dc=franksdank,dc=cloud` +
+`(memberOf=cn=jellyfin_admins,ou=groups,dc=franksdank,dc=cloud)`. Deliberate
+today: the server's own admin account stays **local**, which is the break-glass
+door when authentik or the outpost is down. A *malformed* admin filter is not
+inert — the plugin throws for every login, so this knob is all-or-nothing.
 
 **Gotcha — the bind credential is a PASSWORD, not a token.** The LDAP bind is
 verified by authentik's default authentication flow (standard-password
@@ -91,10 +242,23 @@ password and the bind fails with `flow error password: Invalid`. The
 service-account password is stored only in the Jellyfin plugin config (the
 `LdapBindPassword` field) — it is never committed to this repo.
 
-**Verification (2026-09-25):** `ldapsearch` bind + the `jellyfin_users`
-search both succeed from Dankclaw; a real user (`alex`, in `jellyfin_users`)
-logged into the web UI with their authentik password and the plugin created
-the matching Jellyfin user.
+**Verification (re-taken 2026-11-04, from Dankserver):** the container reaches
+`192.168.0.143:389`; the service-account bind plus the `jellyfin_users` search
+return every member; LDAP-Auth 23.0.0.0 loads on Jellyfin 12.4; the `alex` row
+still carries
+`AuthenticationProviderId=Jellyfin.Plugin.LDAP_Auth.LdapAuthenticationProviderPlugin`.
+
+**Law — "the authentik login doesn't work" has two opposite causes; read the
+Users table before touching authentik.** Copy `config/data/jellyfin.db` with its
+`-wal`/`-shm` and read it read-only (never sqlite the live file):
+`Users.AuthenticationProviderId` names the door — `…Users.DefaultAuthentication
+Provider` = local password, `…LDAP_Auth.LdapAuthenticationProviderPlugin` =
+authentik. Because the LDAP search is scoped to `jellyfin_users`, a name that
+exists only as a local account is rejected by the plugin and Jellyfin then
+silently falls back to the local provider: same UI message, opposite fix. The
+instrument for "did a bind ever reach authentik" is the **event registry**, never
+the outpost journal — see [[Authentik Outpost Reference]].
+
 **Gotcha — the login endpoint changed in Jellyfin 12.1.** The legacy
 `POST /Users/Auth/Password` route (and its `LegacyEmbyRouteRewriteMiddleware`)
 was removed; probing it now returns a silent 401 with no plugin log, which
@@ -105,12 +269,220 @@ where the LDAP plugin runs. Verified 2026-09-25: `alex` authenticates over
 `AuthenticateByName` (HTTP 200; full plugin flow logged — service bind,
 search, `ldapUid`, user bind, token).
 
+## Metadata providers and their keys (2026-10-30)
+
+TMDb, OMDb, StudioImages and MusicBrainz are **core plugins** in Jellyfin 12 —
+no DLL, just a configuration file the running container owns under the data
+root. TVDB is not core (its metadata provider was removed in 10.9), so it is an
+official-catalog plugin.
+
+| Provider | Shape on disk | Key lives in |
+|---|---|---|
+| TMDb | `config/plugins/configurations/Jellyfin.Plugin.Tmdb.xml` → `<TmdbApiKey>` | `/home/dank/Containers/jellyfin/provider-keys` → `TMDB_API_KEY=` (mode 600) |
+| TVDB | `config/plugins/thetvdb_24.0.0.0/` + `…Jellyfin.Plugin.Tvdb.xml` → `<ApiKey>` | the same file → `TVDB_API_KEY=` |
+| OMDb | `…Jellyfin.Plugin.Omdb.xml` | **not configured** — its key is issued by email, 1 000 req/day, and its ToS forbids storing the returned data; ratings stay Jellyfin's own lookup |
+
+**None of it is tracked in git.** Those XML files belong to the container and
+hold credentials, and `tests/jellyfin_providers.bats` fails the suite if one is
+ever committed. The keys themselves sit in the per-service host env file that is
+the fleet's convention for container credentials (grafana, caddy, pihole,
+copyparty, romm, seerr): `provider-keys`, mode 600, beside the data root.
+`scripts/jellyfin-providers.sh` is the only thing that writes the XML:
+
+```
+scripts/jellyfin-providers.sh plan      # what would change; prints no key
+scripts/jellyfin-providers.sh install   # plugin + keys, previous copy to trash
+scripts/jellyfin-providers.sh verify    # plugin + key LENGTHS, never key bytes
+```
+
+- **A key never enters git, argv, or the tool's output.** The installer reads
+  that 600-perm file itself (a value in argv is world-readable through
+  `/proc/<pid>/cmdline`) and refuses it when it is group- or world-readable —
+  an exposed key is an error, not a warning. A key that shows up in a transcript
+  or `ps` output has leaked: rotate it with the provider, rewrite
+  `provider-keys`, re-run `install`.
+- **The TVDB artifact is pinned three ways**: sha256
+  `6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319`, the
+  official plugin guid `a677c0da-fac5-4cde-941a-7134223f14c8`, and version
+  `24.0.0.0` (`targetAbi 12.0.0.0`, which fits the deployed 12.4.0). A sha
+  match alone only proves the bytes are the bytes fetched — the guid check is
+  what says it is the official plugin. Rollback: move `thetvdb_24.0.0.0/` into
+  the trash and revert the pin; a second `thetvdb_*` directory is a refusal,
+  never a quiet second copy.
+- **The homelab TVDB key is a licensed key** (its JWT claims say
+  `community_supported: false`), so it needs **no PIN** — the optional
+  `TVDB_PIN=` line exists only for a subscriber-shaped key.
+- **TVDB is installed but is not a metadata agent**: TMDb still identifies
+  everything. Enabling TVDB as a provider would re-fetch show metadata across
+  the library, so it is a deliberate, separate change.
+- **A plugin owns its own config file, under its own root, and omits null
+  settings** — so the element an installer wants to fill may simply not exist,
+  and its root is `<PluginConfiguration>`, not `<Configuration>`. TVDB's file had
+  no `<ApiKey>` element at all, which is why `install` refused with
+  `no place to write <ApiKey>` from October until the anchor fix: the secret sat
+  valid in `provider-keys` the whole time, and `verify`'s `MISSING key` was this
+  fault, not a missing key. The writer now anchors on the file's own root and
+  still refuses by name when a file has none.
+- **The deployer places the keys before it restarts jellyfin**
+  (`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
+  failing when `provider-keys` is absent, so a host without it still deploys.
+- **Keys and plugins are read at start-up**: after a manual `install` the
+  change is inert until `./scripts/deploy.sh dankserver --service jellyfin`.
+- TMDb's terms are the reason this stays narrow: attribution required, no
+  caching beyond 6 months, no derivative works. Fine for a personal homelab,
+  and the reason no bulk enrichment pass runs against it.
+
 ## Configuration workflow
 
 1. Create libraries pointing at `/data` folders (movies, shows, audio).
 2. Set metadata/language preferences (`LANG=en_US.UTF-8` pinned in the unit).
 3. Configure clients to use `http://192.168.0.235:8096` or the guarded vhost.
 
+## Triggering a scan
+
+Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
+trigger`, then `… wait`. It calls the library settled only on the rule below — the
+task seen `Running` (or a run that finished while it watched) first, then
+`--require-quiet` reads with no scan and no writes — and refuses by name when the
+scan never started, because a watcher that saw no `Running` would otherwise report
+the pre-scan library as the fresh one. `… status --after <the trigger's epoch>` is the
+"has the scan reached my moves" answer, read from the task's `EndTimeUtc` — and that
+timestamp only advances when a scan *finishes*, so the epoch must be one your trigger
+could not have followed: a journal mtime is already passed by whatever scan ran before
+you asked, and bare `status` cannot tell that walk from yours, while `wait` refuses such
+an epoch by name. The Plex door is worse still — [[Plex]].
+
+It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
+config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
+value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
+`Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
+every key, valid or not — never as a header argument, because
+`/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
+`BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
+quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
+stock `~/.local/share/jellyfin`.
+
+`204` is acceptance; anything else queued no scan, so re-issue the identical
+request instead of diagnosing. A `401` from a key that answers `200` on a
+hand-typed request is this block's quoting, not the server: bash `printf`
+collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
+receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
+204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
+key against the server's own table — `select count(*) from ApiKeys where
+AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
+the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
+log, see [[Media Relayout]]). Reading progress with the same config trick takes
+**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
+transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
+
+One accepted refresh is the whole procedure. A second one while a scan runs moves
+`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
+later `204` is accepted and then discarded — the status code stops meaning
+anything. The fix is `systemctl --user restart jellyfin.service`, one POST after
+health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
+response code.
+
+**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is
+the progress read.** A task that has never completed answers
+`LastTaskResult=None, EndTimeUtc=None`, and that pair is what separates "never
+ran" from "finished a moment ago" — the difference between a door that is dead
+and one that merely answered `204`.
+
+**The settle read is `GET /ScheduledTasks`: the row whose `Name` is exactly
+`Scan Media Library`, and its `State` (`Idle`|`Running`).** `/Library/ScanStatus`
+is a 404 on 12.4, so a watcher built on the old endpoint reports nothing and a
+scanning server looks an idle one. Call it settled only after a `Running`
+sighting followed by two consecutive `Idle`s: sub-tasks leave gaps that a single
+poll reads as finished, and a census taken in one reports the pre-scan library.
+
+**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider
+switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before
+the flip and again after it, the call answered `204`, the journal wrote zero lines
+over the following ten minutes, and `Scan Media Library` kept its never-run
+signature; a service restart changes nothing. `POST /Library/Refresh` does run —
+`204`, then `Running` inside 15 s at 33 % CPU — but it walks every library, so on
+this build a library-scoped metadata pass has no working door and the global one
+is the only lever.
+
+**That endpoint takes its parameters from one source.** The documented query form
+(empty body) and the JSON body form (no query) each parse; supplying both at once
+answers `400` with an empty body whatever the values, and `ValidationFailure` is
+rejected in the body while the query form accepts it.
+
+**`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress
+read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The
+record's useful field is `ItemId`, and the live library set is still the entries
+whose `Locations` are non-empty.
+
+`inotify` does not cross a rootless podman bind, so folders created on the host
+stay invisible until a pass like this runs; every pass that writes into a library
+ends by firing one.
+
+**`Movies` has no watcher at all.** `EnableRealtimeMonitor` is per library (table in
+*Data layout*) and it is false for `Movies` and `Collections`. Nothing notices a
+folder created under `/data/Movies`: it stays invisible until a recursive refresh
+names it. So every pass that writes into a library ends by firing one, and "Jellyfin
+does not see the file I just moved in" is the designed answer, not a fault to chase.
+
+## API-key ledger and rotation
+
+`/Auth/Keys` is the ledger. `GET` lists every key **with its `AccessToken` in
+cleartext**, so an admin token in hand prints the whole fleet's keys;
+`POST /Auth/Keys?app=<name>` mints one and answers `204` **with no body** — the
+new token is read back from the `GET`, never from the create response.
+`DELETE /Auth/Keys/{token}` revokes **by the token string**, and a parameter that
+matches nothing answers `204` too, so `204` there is not proof anything was
+removed. Discover a route's verb from the `Allow` header of an `OPTIONS` (or
+`GET`) response — never by firing `DELETE` at a guess.
+
+- `IsActive` reads `False` on rows that authenticate `200`. It is not a usability
+  signal; only an authenticated request is.
+- `DateLastActivity` never moves off `0001-01-01` on 12.4, so "is this key in
+  use?" cannot be answered from the ledger. The census is repo + host credential
+  stores + the `Devices` table (whose `DateLastActivity` *is* real).
+- Rotation order: mint → rewrite **both** stores (`~/.config/secrets/jellyfin`
+  and `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) → prove the new key on
+  `/System/Info` → `DELETE` the old token → prove `401`. Revoking while a store
+  still holds that value breaks the scan door silently.
+- Blast radius as measured 2026-11-03: no repo script reads `JELLYFIN_API_KEY`;
+  Seerr keeps a key of its own in its own DB (which of its two rows it uses was
+  never confirmed), so revoke by token and never by app name.
+
+## Big-screen theme (the TV)
+
+`dankserver/jellyfin/theme/wmc/theme.css` is repo-owned and bind-mounted over the
+image's **Media Center** theme slot (the `Volume=` line in
+`dankserver/jellyfin/quadlet/jellyfin.container`). It exists for
+`danklivingroom`, which renders this server-served web app at 3840×2160 with no
+client-side scaling, so 14 px text arrives on a panel read from three metres.
+
+The file is four rules, and its first line is the design:
+`@import url("../dark/theme.css")` pulls in the **live upstream** dark palette, so
+the repo never vendors a CSS snapshot and a Jellyfin image bump cannot strand one.
+That works because jellyfin-web applies a theme as
+`<link rel="stylesheet" href="themes/<id>/theme.css">` — a relative URL resolved
+against the CSS file, not the document. The size lives in one number:
+`--dank-big-screen-zoom` (1.75 ≈ three metres; 1.5 mild, 2.0 projector). `zoom` on
+the root element is the right lever: the card-grid code measures layout pixels, so
+fewer, larger columns are computed rather than overflow produced.
+
+- **Adoption is per device, and it stays that way.** The theme a client renders
+  comes from its own settings (localStorage first), so there is no server-side
+  "everyone gets the big UI" knob. On the screen that wants it:
+  **Display Settings → Theme → Media Center.** That is also the honest label —
+  the dropdown names are compiled into the bundle, so a "Big Screen" entry will
+  never appear; do not go looking for one.
+- The mount is `:ro` and the app only fetches the file, so the failure mode of a
+  stale `@import` target is slightly wrong colors, never a dead web app.
+- **Editing the CSS needs no restart.** The bind hands out the host file's live
+  bytes, so a changed zoom number is on screen at the next page load. The planner
+  agrees by construction: only `dankserver/<svc>/quadlet/**` maps to a service, so
+  a `theme/**`-only commit plans nothing — that is correct, not a gap to "fix",
+  because restarting a media server for a font size is the wrong trade. Only the
+  first deploy of the `Volume=` line restarts anything.
+- **Rollback is one line:** drop the `Volume=` and redeploy; nothing in the
+  database records the theme, so no cleanup follows.
+
 ## See Also
 
 - [[Plex]] — co-resident media server on the same volume

serverAppConfigs-docs/Dankserver/Media Audiobook Relayout.md

@@ -0,0 +1,78 @@
+---
+type: reference
+title: Media Audiobook Relayout
+description: "Re-filing /mnt/entropy/Audiobooks by Plex's chapter rows: narrator shelves, no renumbering, the refusals."
+tags: [dankserver, media-server, plex, media-relayout]
+timestamp: 2026-10-31T00:00:00Z
+---
+
+# Media Audiobook Relayout
+
+`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
+at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
+of it as Artist → Album → Track: the album is the BOOK, that book's single artist
+is the NARRATOR, both matched out of the files' own tags. [[Audiobookshelf]]
+streams the same tree, read-only, and owns nothing in it. Today one book is split
+across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
+pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
+re-files the tree so the property both servers' book scanners assume is true —
+
+    Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
+
+The shared gates, the executor, the journal law and the rollback live in
+[[Media Relayout]]; this page is the audiobook arm only.
+
+```bash
+# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
+python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
+    --out /tmp/ab_rows.tsv
+# 2. What is on disk (stat only).
+python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
+    --out /tmp/ab_enum.tsv
+# 3. The plan.
+python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
+    --enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
+    --exceptions /tmp/ab_exceptions.txt
+# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
+#    as [[Media Relayout]] shows, on its own journal, and refresh the Plex section
+#    as in step 5 of the movie Procedure — from inside the container, never over
+#    the host port.
+#    Audiobookshelf stores absolute paths, so the same refresh has to reach its
+#    library too ([[Audiobookshelf]]) — otherwise it keeps rows for folders that
+#    no longer exist.
+scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
+    --journal /mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl --apply
+```
+
+The group is the identity Plex asserts in its own database — **(narrator, book,
+year)** — so one book arriving in four part folders becomes one folder, and a
+1965 and a 2020 recording that share a title stay two books.
+
+Two laws are forced by what the rows actually hold:
+
++- **A chapter is never renumbered.** Most books contain chapters whose index Plex
+  does not know, so a number written into a name would be invented — and it would
+  buy nothing, because both servers order playback by the embedded track tag, not
+  by the file name. A wrong number in a name is the number a human later trusts.
+  Fix order in the tags: a Plex scan re-reads tags and never renames a file.
++- **The shelf is the narrator, not the author.** Every book has exactly one artist
+  row and it is the narrator; the author survives only inside some album titles
+  (`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
+  mean parsing titles where a ripper wrote one and inventing it everywhere else.
+  The author stays visible in the folder name Plex itself shows.
+
+What it refuses, the music arm's list with one addition: a video container, audio
+no chapter row names, and a loose file in a folder holding two books all park under
+`_unsorted` with the path they have today — and so does a chapter whose row names
+**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
+that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
+`Various Artists` is not a placeholder — that is a multi-cast read.
+
+The reviewed plan lives at `/mnt/entropy/.media-campaign/media-relayout-audiobooks-plan.tsv`; it
+applies on its own journal, behind the music writer, and then refreshes the Plex
+section. Verify it with `--db-kind plex` — Jellyfin holds no rows here, so a
+Jellyfin pass proves nothing either way.
+
+## See Also
+
+[[Media Relayout]], [[Media Music Relayout]], [[Plex]]

serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md

@@ -0,0 +1,302 @@
+---
+type: reference
+title: Media Duplicate Prune
+description: "Find and hold duplicate films and series on the array: the screen, per-episode rule, guards, purge."
+tags: [dankserver, media-server, storage, procedure]
+timestamp: 2026-11-02T01:00:00Z
+---
+
+# Media Duplicate Prune
+
+The movie library holds the same film twice in two different ways — the same file
+downloaded again, or the same film grabbed a second time at another quality — and
+[[Entropy Volume]] is permanently full, so both are worth finding. They need
+different rules, and neither is decided by name similarity alone.
+
+A series duplicates differently again: one show spread over several folders, so
+neither the file nor the folder is the unit — see *A series is measured per
+episode*.
+
+`/mnt/entropy/media-park` was a third kind: a parking lot that held whole shows
+and films nothing served. It is gone; its contents went to `NeedsProcessing`.
+
+## What counts
+
+| Class | Test | Action |
+|---|---|---|
+| Byte-identical | equal size, equal content hash | keep one copy, hold the rest |
+| Same work, worse encode | same title, same year, every runtime within 5 % | keep the largest — Frank's rule |
+| Different cut | same title and year, runtimes more than 5 % apart | **never touched**: an extended and a theatrical version are different products |
+| Same words, different film | title matches, year or duration does not | never grouped — this is the false-merge class that would destroy a unique file |
+| Series split across folders | same show by tvdb id, then the episode `(S,E)` — never the folder name | keep the largest carrier of each episode (`scripts/media-tvdupes.py`) |
+
+## The screen
+
+The array cannot afford a full-content read, so identity is established in steps
+and each step only narrows what the next one touches:
+
+1. `scripts/media-census.py enumerate` — size is free, and identical files share it.
+2. Head+tail `blake2b`, 4 MiB at each end, on the size collisions only.
+3. **Before anything moves**, confirm with a sparse-window hash: 32 MiB at eight
+   evenly spaced offsets. Re-hashing a remux-heavy set in full read 150 GiB for
+   the better part of an hour; the sparse window cost 50 s, a file that merely
+   shares its head and tail cannot survive it, and the destination is a
+   recoverable hold rather than a deletion.
+4. Durations come from `scripts/media-census.py probe` — the relayout's own
+   census. Do not hand-roll a second probe: it re-seeks the array for an answer
+   the repo tool already produces, and a probe is the one census step that costs
+   hours instead of seconds.
+
+## Which copy survives
+
+In order: not one inside an `extras`, `featurettes`, `samples` or `trailers`
+folder (an extra is not a copy of the film, and extra names collide across films);
+then the shallowest path; then the file whose name equals its `Title (year)`
+folder; then the largest. For the same-work class the largest is the answer, and
+the canonical-name rule only breaks a tie — without it a `Film - 1080p.mkv` can
+outlive `Film.mkv` for no reason but a space sorting before a dot.
+
+## A series is measured per episode
+
+`scripts/media-tvdupes.py` censuses a TV library and writes a staging plan in the
+executor's own format; it never moves anything itself.
+
+```bash
+python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
+    --hold /mnt/entropy/.AGENTTRASH/<date>-tv-duplicates \
+    --meta /var/lib/media-meta/view/TVShows --witness witness.tsv \
+    --ids /mnt/entropy/.media-campaign/tv-tvdb-ids.tsv \
+    --out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv --json /tmp/tv-groups.json
+# then media-apply.py apply / verify / sweep / rollback, exactly as for a film prune
+```
+
+- **Identity before geometry.** A folder's tvdb id is read from its `[tvdbid-N]`
+  name tag, from the `tvshow.nfo` the [[Media Metadata View]] exposes (`--meta`),
+  and from a `folder<TAB>id` map (`--ids`). Build that map from Jellyfin's own
+  match, not a lookup: `BaseItems` (`Type like '%TV.Series'`) joined to
+  `BaseItemProviders` where `ProviderId = 'Tvdb'`, the `/data/TVShows/` prefix
+  stripped to the bare top-level folder name — 616 folders, none carrying two
+  ids, written to `/mnt/entropy/.media-campaign/tv-tvdb-ids.tsv`. Two folders with disjoint ids
+  are two shows however alike they read — *Avatar: The Last Airbender* (1999)
+  beside the 2024 remake, *Millennium* (1996) beside a German thriller Jellyfin
+  had also titled *Millennium* — and a pair whose identity cannot be proven is
+  reported `UNRESOLVED` and left alone. The fix is the tag, never a guess.
+- **A tag outranks every derived id.** The `--ids` map and the `tvshow.nfo` layer
+  fill an untagged folder and are **dropped** when they contradict its own
+  `[tvdbid-N]`, which the report prints as `DISAGREE`. Measured the same day: the
+  provider table carries a 2026 special's id on `Malcolm in the Middle (2000)
+  [tvdbid-73838]`; unioned, those folders are one show and the special's bigger
+  files dominate the original's S01E01–E04 — 3.2 GiB of the 2000 series bound for
+  the hold as "worse copies". The pass that staged 54 GiB ran with no `--ids`, so no
+  staged file rests on a borrowed id.
+- **A folder-level verdict destroys content.** Of 25 structural candidates in the
+  2026-11-02 census, **10 were complementary halves**: *Game of Thrones: The
+  Complete Series* holds no S01 and no S08, `Modern Family S05` is the only copy of
+  season 5, *Loiter Squad Uncensored* the only copy of season 1. Judged per folder
+  the pass would have staged about 340 GB of unique episodes; judged per episode it
+  staged 54 GiB of true copies.
+- **A file is droppable only when every episode it carries is held elsewhere in the
+  same show at >= its size** (Frank's rule, applied per episode). A byte-identical
+  pair ties and the tie breaks by path so exactly one copy stages; a file holding
+  one unique episode stays whatever else it carries.
+- **A bare number is never an episode.** `256383.mp4` may be an episode id, a
+  poster or a whole season, so key-less names are never staged and are counted in
+  the report (4,036 after the 2026-11-02 pass, almost all genuine bonus material).
+ Where a packager named files by id, Jellyfin's own table is the witness: a
+ `path<TAB>season<TAB>episode` TSV from `BaseItems` where `Type like '%TV.Episode'`
+ and both index numbers are set, read `mode=ro&immutable=1`, its `/data` prefix
+ rewritten to the raw array path. Jellyfin 12 keeps `Path`, `ParentIndexNumber` and
+`IndexNumber` on `BaseItems` and stores the full CLR type name — confirm with
+`pragma table_info` before trusting the query, because an empty witness silently
+falls back to name parsing.
+  Seven clips in the library are still named only by number, and where Jellyfin
+  was asked its own `Name` for them is the number too, so no database holds a
+  better title — that residue is naming, not a task.
+- **A pack states a range and competes on all of it.** `S01E01-02` is droppable only
+  when both halves are held elsewhere, and it wins the survivor seat for both when
+  it is the largest carrier: the byte law compares files, and a name both servers
+  parse as a range stays indexed as two episodes.
+- **A sidecar that shares the doomed copy's stem goes with it** — subtitles, `.nfo`,
+  artwork keyed on the stem — so a staged duplicate leaves no orphan beside the
+  survivor. The 2026-11-02 hold is 220 media files and nothing else: the dominated
+  copies were bare rips, and counting a hold's non-media files is the census that
+  tells you which case you are in.
+- **The audit gates the plan file.** Every survivor is re-statted and every
+  destination re-checked before `--out` is written; on failure the report prints
+  `AUDIT FAILED` and writes no plan, so there is nothing to hand the executor.
+- **Two folders that look duplicated and share zero episode keys are complementary,
+  not duplicated.** That is how the two pairs the census still refuses to merge were
+  settled without touching them: *Star Wars - The Clone Wars* (12 files, 1.3 GiB)
+  beside *Star Wars The Clone Wars* (121 files, 214 GiB), and the *Batman: The
+  Animated Series* leftovers (3 files) beside the tagged folder (85) — `shared
+  episode keys: 0` in both cases, so a tag would have freed nothing and cost a
+  rename with library consequences. Re-running the whole census with the id map over
+  all 753 top-level folders (606 already tagged) staged **0 files**: the library is
+  duplicate-free to the limit of provable identity, and what remains is naming.
+
+## Guards
+
+Each of these is a file that would have been destroyed without it:
+
+- **`.!qB` is qBittorrent state, not a duplicate.** It can be byte-identical to
+  the finished file while qB still counts the download incomplete; moving it
+  breaks the re-check and re-downloads the film. Report it, leave it. Both
+  leftovers in `Movies` are 13 months stale, so they are Frank's purge decision,
+  not a prune decision.
+- **Require a year, and keep digits in the title.** Stripping digits merged
+  *Akira — Teaser 1* with *Teaser 2*; without a year, two different extras that
+  happened to share a name merged.
+- Both members must be feature length, and a name carrying `disc`, `cd` or `part`
+  is excluded: half a multi-disc release is not a copy of the other half.
+- Media containers only. Grouping every extension buries the report in subtitle
+  and cover pairs — and a duration guard that appears to fire on 720 groups is
+  counting subtitles, not protecting films.
+- **A short file named after a film is promotion, not a movie.** Thirty-two files
+  in `Movies` — 31 clips and the `.nfo` filed beside one of them — carried real
+  titles (`The Descent (2005) - SD.mkv`) and were 8–20-second h264 video at
+  640×360, 1024×576 or 1280×720, 49 MiB all together —
+  25 of them in a `_release-group watermarks/` folder and the rest at the library
+  root or inside one film's folder. No duplicate screen ever catches them: nothing
+  shares their size, and the feature-length guard above makes them invisible to the
+  similar-title pass too. They were held as `.AGENTTRASH/20261104-promo-clips`
+  (journal `/mnt/entropy/.media-campaign/promo-hold.jsonl`, which survives) and purged
+  the same evening, once both databases read `REAL: 0` against that journal.
+  The census that finds the next batch is a duration floor, not a hash: a video under
+  a minute on a name that reads like a feature is advertising.
+
+## The hold
+
+Nothing is deleted. Every doomed copy moves into a dated hold with the same
+executor the relayout uses, so any file comes back:
+
+```bash
+scripts/media-apply.py apply   --plan /mnt/entropy/.media-campaign/dup-prune-similar-plan.tsv \
+    --root /mnt/entropy --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl          # dry
+scripts/media-apply.py apply   --plan … --root /mnt/entropy --journal … --apply
+scripts/media-apply.py verify  --plan … --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl
+scripts/media-apply.py sweep   --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
+scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
+```
+
+The plan rows are the tool's own format (`action src dst kind`, `mkdir` rows for
+every hold parent), and the planner re-checks the two things that go stale between
+analysis and apply: a file that has moved since the census, and a group whose
+keeper is missing — either one drops the whole group rather than half-applying it.
+
+- **A hold on the same volume frees nothing.** The purge is a separate, approved
+  act and the only thing that moves the free-space number; `df` does not budge
+  when 1 TiB moves sideways.
+- `sweep` removes the release folders the pass emptied. Empty folders dated a year
+  ago are torrent detritus (`Subtitle,info`, `Screens`, `Sample`) from a previous
+  era, not this pass — read the mtimes before blaming the prune.
+- The 2026-11-01 pass held 226 files (~1.1 TiB) in
+  `/mnt/entropy/.AGENTTRASH/20261101-duplicate-prune`, journals
+  `dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`.
+- The 2026-11-02 TV pass held 220 files (54 GiB) in
+  `/mnt/entropy/.AGENTTRASH/20261102-tv-duplicates` — plan `tv-dupes-plan.tsv`,
+  journal `tv-dupes.jsonl`, 220 moves / 16 hold folders / 7 husks swept — and the
+  same day's TVShows detritus pass staged 109 files (552 KiB) in
+  `/mnt/entropy/.AGENTTRASH/20261102-tvshows-detritus` with plan
+  `tvshows-detritus-plan.tsv` and journal `tvshows-detritus.jsonl`. All four
+  journals and plans live at the `entropy` root, not `/tmp`.
+- **Purged 2026-11-04, on Frank's word, after the gate below:** the duplicate
+  prune, the TV duplicate pass, the Ghibli duplicates (6 files — both `Nausicaä`
+  copies among them) and the 2026-11-04 promo-clip hold left the array trash,
+  ~1.2 TB freed (the array went 218 GB → 1.4 TB available). The journals survive as
+  the only record of where a purged file came from; rollback after a purge is
+  impossible, which is why the gate runs first.
+
+### The purge gate: a carrier for every held file
+
+`REAL: 0` in both databases says only that no row points at a moved path; it says
+nothing about whether the film survived. So a purge waits on a stronger proof:
+**every held file has a same-work carrier in the live library at >= its size,
+re-derived at purge time**, never copied from the plan. The 2026-11-04 purge re-found
+all 452 held files this way; the first pass of the gate could not place 142 of them,
+and every one of those was a matcher gap, not a lost film. What the re-derivation has
+to absorb:
+
+- **The layout moves under the plan.** The relayout renames folders to
+  `Title (YYYY)` and `[tvdbid-N]/Season N` and renames the keeper files too, so the
+  paths *and* the sizes a plan recorded no longer describe the library. Keys are work
+  identity — normalized title+year for a film, `(S,E)` for a TV pack — never a path.
+- **Fold accents, then compare.** The library holds `Nausicaä`; a plan, a
+  `find -ipath "*Nausicaa*"`, and an ASCII substring search all miss it. NFKD-fold and
+  drop combining marks, or a real keeper reads absent for two rounds and the file
+  beside it starts to look like an orphan.
+- **The carrier is often another container, and usually larger.** A held `.mp4` can be
+  kept as the sibling `.mkv`; keep-larger means the keeper is frequently far bigger (a
+  377 MB held episode matched a 501 MB keeper; a 1.6 GiB film stands beside a 34 GiB
+  BDRemux of itself in another folder). "Same folder, same bytes" reports survivors as
+  lost — it is the wrong test.
+- **An exact twin can sit in a sibling folder**, so "no same-size file in the folder"
+  needs an array-wide name search before it means anything.
+
+### Running the purge
+
+`scripts/purge-agenttrash.sh` is the only door — never `rm`. It acts on exactly one
+object: a root's own `.AGENTTRASH`. A partial purge is therefore **staged, not
+filtered**: move the approved buckets into a scratch root
+(`.PURGE-<date>/.AGENTTRASH/`) and point the purger at that root, and park anything
+not approved at the array root as `.HOLD-<date>-*`, out of its reach. Dry-run first —
+the plan must print exactly one `plan empty` row, and it is the same row `--execute`
+acts on — and send `--log` into `.media-campaign/`, because `/tmp` is not a record.
+Then read `df`: a purge is the only media act that moves the free-space number.
+
+## After the pass
+
+Both servers still hold rows for files that are now in the hold, and neither
+notices: the media bind is the [[Media Metadata View]], and array changes do not
+cross it as inotify events. Scan in the order the rescan law sets — Jellyfin to
+`Idle`, then Plex's Movies section alone, one accepted refresh each; the requests
+are owned by [[Jellyfin]] and [[Plex]].
+
+The Plex leg may be deferred on purpose: stop **and mask** Plex (`systemctl --user
+stop plex && systemctl --user mask plex`) so its metadata fetches cannot stack on a
+live Jellyfin scan, and because a masked unit stays stopped through a reboot —
+`unmask` before starting it. The 2026-11-01 morning ran it once Jellyfin reached
+`Idle`: one accepted `refresh` on the Movies section cleared every row that named a
+held path. The 2026-11-02 TV pass did the same on the TV Shows section (key 3): one
+accepted refresh, then `media-verify.py` against that journal reads `landed: 0`,
+`still at src: 0`, `REAL: 0` on **both** servers — the correct end state for a prune.
+
+The same evening's detritus pass is a second, separate hold: 423 files (12 MiB) in
+`/mnt/entropy/.AGENTTRASH/20261101-detritus-sweep`, plan `detritus-sweep-plan.tsv`,
+journal `detritus-sweep.jsonl`, 542 husks removed by `sweep`. It moved files only
+from folders whose whole subtree held no media, so it creates no stale rows and
+needs no scan. The guards that hold a folder back are the finding: external
+subtitle folders (428 of them, 6 GiB) and ripped Blu-ray streams — `.sup` PGS,
+`.thd`/`.eac3`/`.dts` audio, `.cci` captions, 25 GiB — are not trash, and an
+extension the census does not recognise is a reason to stop, not to sweep.
+
+A folder holding a `.plexmatch` is held back even when its subtree looks empty: that
+file is a manual match — a human decision about what the folder is — and detritus
+rules do not overrule one.
+
+`scripts/media-verify.py` reads differently for a prune than for a relayout:
+`landed: 0` is the **correct** end state (no server may index the hold), so the
+verdict is `still at src: 0` with `REAL: 0`, and the rows that simply vanished
+from the database are the pass succeeding.
+
+One residue is normal, so do not chase it: a section pass rewrites the feature rows
+first, and rows for clips in a film's `other/` or `extras/` subfolder can still name
+a held path after the section's `refreshing` flag is back to `0` — and a second
+section pass does not clear them either, so do not run one hoping for `0`. Beside
+`REAL: 0` that is a dead disc-menu clip — not a false prune, and no reason to touch
+the cache, the database, or the hold.
+
+## Failure signatures
+
+| Symptom | Meaning |
+|---|---|
+| A size group whose members split under the sparse hash | The head+tail screen collided. Those files stay exactly where they are and get reported. |
+| The planner holds far fewer files than the analyzer counted | Two implementations of "largest wins". The analyzer must emit `keep`/`hold` marks and the planner only transcribes them — a planner that re-derives the winner from the hold list alone drops the keeper out of the group and prunes almost nothing. |
+| `df` unchanged after a TiB of moves | Expected: the hold is on the same filesystem. Only the purge frees space. |
+| A film that "disappeared" after a prune | It is in the hold, named by the journal: `media-apply.py rollback --journal … --apply`. |
+
+## Links
+
+- [[Media Relayout]] — the census tools, the executor, the journal format.
+- [[Media Metadata View]] — why neither server sees the change without a scan.
+- [[Jellyfin]] · [[Plex]] — the scan requests, one server at a time.
+- [[Entropy Volume]] — the array that is always full.

serverAppConfigs-docs/Dankserver/Media Metadata View.md

@@ -0,0 +1,180 @@
+---
+type: reference
+title: Media Metadata View
+description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
+tags: [dankserver, media-server, jellyfin, plex, storage]
+timestamp: 2026-11-02T01:00:00Z
+---
+
+# Media Metadata View
+
+`/var/lib/media-meta/view` is an OverlayFS merge of the media library: the RAID6
+array below, a directory on the NVMe above. It exists so [[Jellyfin]] can write
+Kodi-format sidecars (`movie.nfo`, `tvshow.nfo`, `episode.nfo`, artwork) that
+[[Plex]] can read. Sidecar metadata is only useful sitting next to the media
+file, and the array is read-only and slow — so the media bytes stay there and the
+*writes* land on SSD. One tree to both servers, one copy of the media.
+
+The mount point is not a volume and holds nothing while unmounted. It sits inside
+`/var/lib/media-meta` so that layers, kernel bookkeeping, the reap trash and the
+merged door are one directory — and so the mount table grows no new `/mnt` entry
+next to the array.
+
+**Deployed on Dankserver — 2026-10-31, both servers.** `plex` binds
+`/var/lib/media-meta/view:/mnt/entropy:ro` and `jellyfin` binds
+`/var/lib/media-meta/view:/data:rw`; `findmnt` reports the overlay with
+`lowerdir=/mnt/entropy upperdir=/var/lib/media-meta/upper`, and each container
+was proven to resolve a real media file through it — `podman exec <svc> ls -la
+<its data path>` is the whole proof, because both servers keep the paths their
+databases recorded. Two service-scoped deploys, each gated by
+`media-view.sh verify` and the Playwright suite.
+
+Re-verified 2026-11-01: `mount` shows the overlay on
+`/var/lib/media-meta/view` (`lowerdir=/mnt/entropy`,
+`upperdir=/var/lib/media-meta/upper`), and the installed quadlets under
+`~/.config/containers/systemd/` carry the view binds:
+`Volume=/var/lib/media-meta/view:/data` for [[Jellyfin]],
+`Volume=/var/lib/media-meta/view:/mnt/entropy:ro` for [[Plex]]. Prove it in that
+order: `scripts/media-view.sh verify` (the deploy gate), then
+`grep -n Volume ~/.config/containers/systemd/{jellyfin,plex}.container`.
+
+Moving a bind means restarting the server, and **a restart discards that
+server's in-flight library scan** — Jellyfin's 7 h 25 m walk ended
+`"Status": "Aborted"` in `data/ScheduledTasks/<task-id>.js` and nothing resumes
+it. Renames made on the array while the view was unmounted are invisible to both
+watchers too, so the additive index has to be asked for by hand; the door is an
+API key, and the row in the failure table below says what to check when it
+refuses.
+
+The cutover itself is an ordinary deploy: `deploy.sh dankserver --service
+jellyfin|plex` runs `ensure_media_view` (`install` then `verify`) before it
+restarts the container, so it must not be fired through the middle of a library
+scan. Two consequences are now live rather than theoretical: inotify events for
+array writes do not cross the merge, so any relayout or prune needs a manual
+library refresh; and a move can leave ghost sidecars above, so `reap` runs after
+a relayout — a pass that moves only media files with no sidecars next to them
+leaves the upper layer untouched, and `reap` correctly finds nothing.
+
+## Shape
+
+| Item | Value |
+|---|---|
+| Lower layer | `/mnt/entropy` — the RAID6 array, source of truth, never written |
+| Upper layer | `/var/lib/media-meta/upper` — NVMe; every write lands here |
+| Work dir | `/var/lib/media-meta/work` — overlayfs internals, same filesystem |
+| Mount point | `/var/lib/media-meta/view` — the merged tree both servers bind |
+| Unit (repo) | `dankserver/host/systemd/media-meta-view.mount` — the source of truth |
+| Unit (host) | `var-lib-media\x2dmeta-view.mount` — systemd names a mount unit after its escaped path (`-` becomes `\x2d`); `media-view.sh install` writes the repo file under that name |
+| Installer | `scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap` |
+| Written by | Jellyfin only (Plex binds read-only); nothing else in the fleet touches it |
+
+Both consumers gate their start on the view with
+`ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh` (it waits
+for the mount, then refuses), and `scripts/deploy.sh dankserver --service
+jellyfin|plex` calls `media-view.sh install` + `verify` before it restarts
+either container.
+
+| Consumer | Bind | Mode | Why that path |
+|---|---|---|---|
+| [[Jellyfin]] | `/var/lib/media-meta/view:/data` | rw | It is the metadata writer; its sidecars become SSD bytes |
+| [[Plex]] | `/var/lib/media-meta/view:/mnt/entropy` | ro | Every path in the PMS database was recorded as `/mnt/entropy/...`; only the HOST side of the bind moved, so its database still resolves |
+
+Putting the sidecars inside Plex's reach is not the same as making Plex use them:
+a library parses `.nfo` only with its *Local media information* agent enabled, and
+this change touches no library setting.
+
+## Commands
+
+```bash
+scripts/media-view.sh plan              # print the resolved shape, touch nothing
+sudo scripts/media-view.sh install      # create dirs, install the unit, mount, enable
+scripts/media-view.sh verify            # is the view really the overlay? (deploy gate)
+scripts/media-view.sh status            # mount options + upper-layer size/file count
+scripts/media-view.sh reap              # list ghost metadata (media moved, sidecars left)
+sudo scripts/media-view.sh reap --apply # move ghosts to the SSD trash, never delete
+```
+
+`--upper-base DIR` and `--lower DIR` relocate the layers for `plan` and `reap`;
+the defaults are the host values above.
+
+## Laws
+
+- **The upper layer must sit on non-rotational storage.** `install` and `verify`
+  refuse via `lsblk -dno ROTA` if the upper base lives on the array: an overlay
+  whose upper is the spinners is the original problem with extra steps, and it
+  silently turns every metadata write back into a seek.
+- **A container that binds the array must refuse to start until the mount
+  answers.** Live 2026-10-31: the array's mount and the media containers raced at
+  boot, podman resolved the bind against the pre-mount directory, and both
+  servers came up blind — Jellyfin read an empty library and began
+  `Removing item, Type: "Season"` by the thousand. Two dependencies people reach
+  for do not work here, both measured on that host: `RequiresMountsFor=` in a
+  **rootless** unit is inert (the user manager cannot order against the system's
+  `mnt-entropy.mount`), and `ConditionPathIsMountPoint=` is fail-*dead* — an
+  unmet condition logs `skipped, unmet condition check` with `NRestarts=0` and no
+  start attempt ever follows, so the unit waits for a human. The working guard is
+  `scripts/wait-for-mount.sh` as `ExecStartPre`: it waits out the race, refuses by
+  name after its window, and `Restart=always` carries the unit from there.
+  systemd does not create a mount point, so `install` creates `view` itself;
+  without it the mount simply fails.
+- **Never delete through the view.** A delete writes an overlayfs whiteout
+  (`.wh.<name>`) in the upper layer, which hides the array's copy without
+  removing it — the file reappears the moment the view is unmounted, and the
+  upper layer now carries a landmine. Deletions belong on the raw path
+  (`/mnt/entropy`), which the relayout tool already treats as the only writer.
+  `reap` refuses to remove whiteouts and only warns about them.
+- **A relayout must be followed by `reap`.** Renames happen on the raw path, so
+  sidecars the view already wrote stay at the old coordinates: a ghost folder
+  the servers browse and find nothing in. `reap` lists upper-layer directories
+  with no counterpart below and moves them (with their subtree) into a dated
+  `.AGENTTRASH/` under the SSD base. It is recursive and file-safe: a sidecar
+  that exists only above is normal, so only *directories* are candidates.
+- **File-watcher events do not cross the overlay.** Plex's NFO agent and
+  Jellyfin's real-time monitor see inotify events for writes that land *in the
+  upper layer*, not for changes on the array underneath. After any relayout or
+  externally-written metadata, trigger a library refresh — see
+  [[Media Relayout]] for the refresh procedure.
+- **`stat` through the view can outlive the file.** Measured 2026-11-02 while a TV
+  dedupe pass moved folders on the raw path: the merged tree still answered `stat`
+  at the old coordinates — link count `0`, and `ls` of the same parent no longer
+  listed the name. Overlayfs keeps a dentry for a lower file that was moved
+  underneath it, so a script that tests existence with `[ -e ]` against the view can
+  believe a file its own `readdir` denies. Prove existence by listing the parent.
+  The media servers walk by `readdir`, so ghosts never reach the library — this bites
+  a tool, never a user, which is why no server symptom accompanies it.
+- **Tools that move or measure media walk the raw array, never the view** — every
+  relayout, prune and detritus pass, and every journal it writes. The view is for
+  Jellyfin's writes and Plex's reads only. One deliberate exception: `tvshow.nfo`
+  sidecars live in the upper layer, so a tool reading show identity points at the
+  view (`media-tvdupes.py --meta /var/lib/media-meta/view/TVShows`).
+
+## Failure signatures
+
+| Symptom | Meaning |
+|---|---|
+| `jellyfin.service` / `plex.service` fail with `refused: still not mounted after 90s: …` | The view or the array is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
+| Servers answer, library looks empty, Jellyfin logs `Removing item, Type: "Season"` by the thousand | Blind bind — the container started before the mount and bound the pre-mount directory. `podman exec jellyfin ls /data` against `ls /mnt/entropy` is the whole diagnosis; restarting the container re-resolves the bind. |
+| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
+| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
+| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
+| `install` prints `refused: upper base /var/lib/media-meta does not exist (put it on the SSD first)` | A fresh host. `install` deliberately never creates the base — a mistyped `--upper-base` must not build a metadata tree on the wrong filesystem — so `scripts/media-view.sh ensure-base` makes the documented one, which the deployer calls first. |
+| A media server's scan record reads `"Status": "Aborted"` | Its container restarted (a cutover, an image bump, an OOM) and the walk was discarded; systemd's `Restart=always` restarts the *server*, never the scan. Trigger a refresh — see [[Media Relayout]]. |
+| The Jellyfin API answers 401 to every token header | Almost always the header **form**, not the key: 12.4 authenticates only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for every key, valid or not — and one collapsed backslash in the curl config produces the same 401. `[[Jellyfin]]` owns the block that works. Before concluding anything, prove the key against the server's own table: `select count(*) from ApiKeys where AccessToken = ?` on `jellyfin.db` opened `mode=ro&immutable=1`. |
+| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
+| Stale dentry through the view: `stat` succeeds with link count `0` while the parent's `ls` omits the name | The lower file was moved on the raw path after the view cached its dentry. Nothing is lost; existence through the view is proved by the parent listing. |
+
+## Rollback
+
+Revert the commit that pointed the two quadlets at the view and re-deploy both
+services — they bind `/mnt/entropy` read-only again, which is the pre-view shape.
+The NVMe upper layer can stay: nothing reads it once the binds are back, and it
+holds no state either server needs. Unmounting alone
+(`systemctl stop 'var-lib-media\x2dmeta-view.mount'`) is safe while no container
+binds it, and the array is never modified by any of this.
+
+## Links
+
+- [[Jellyfin]] — the metadata writer; its data layout section.
+- [[Plex]] — the read-only consumer; in-place bind-mount history.
+- [[Media Relayout]] — the tool that moves media on the raw path and the reason `reap` exists.
+- [[Entropy Volume]] — the array underneath.

serverAppConfigs-docs/Dankserver/Media Music Relayout.md

@@ -0,0 +1,68 @@
+---
+type: reference
+title: Media Music Relayout
+description: "Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals."
+tags: [dankserver, media-server, jellyfin, media-relayout]
+timestamp: 2026-10-31T00:00:00Z
+---
+
+# Media Music Relayout
+
+`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
+tagged almost all of it. `scripts/media-music.py` re-files it into the shape
+Jellyfin's music scanner reads —
+
+    Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
+
+— and the authority is the tag row of the track, never the name the ripper gave
+the file. The shared gates, the executor, the journal law and the rollback live in
+[[Media Relayout]]; this page is the music arm only.
+
+```bash
+# 1. The authority: Jellyfin's audio rows, one per track, with the album and
+#    artist ids its own database carries (the DB opens read-only).
+python3 scripts/media-census.py music --root /mnt/entropy/Music \
+    --out /tmp/music_rows.tsv
+# 2. What is on disk (stat only).
+python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
+    --out /tmp/music_enum.tsv
+# 3. The plan.
+python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
+    --rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
+    --exceptions /tmp/music_exceptions.txt
+# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
+#    executor, journal and rollback as [[Media Relayout]] shows, on its own journal.
+scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
+    --journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl --apply
+```
+
+The group is what the tags assert, not where the files sit: **(album-artist tag,
+album, year)**. One album arrives as three disc folders and as two copies of one
+rip, and one MusicBrainz release group is one album. A compilation is one album
+whose tracks name many performers, so it files once under `Various Artists`:
+keying on the performer would file one album once per artist. Two different
+album-artist tags on one album name are two albums, which is what Jellyfin's two
+album items say.
+
+What it refuses:
+
++- **A sidecar is placed by the audio around it, never by its own name.** A
+  `.lrc`, `.txt` or picture that shares a track's stem follows that track and
+  takes the name the track now has, or Jellyfin cannot find it; anything else
+  goes to the album its own folder was placed in, or to the folder whose discs
+  sit one level below it. A folder whose audio belongs to two albums parks its
+  sidecar rather than guessing.
++- **An image album keeps its name.** A folder holding one audio file and a
+  `.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
+  index it as one track.
++- **A video container parks**, because Jellyfin will not index it as music and a
+  rename to `.m4a` would claim it is; **audio no provider row names parks** too,
+  because unmatched audio is a tagging job, not a renaming job.
+
+ONE writer at a time on this array: two rename passes race, and neither journal
+tells. Verify the run against Jellyfin's own tables — see *Proving the move
+landed* in [[Media Relayout]].
+
+## See Also
+
+[[Media Relayout]], [[Jellyfin]], [[Media Audiobook Relayout]]

serverAppConfigs-docs/Dankserver/Media Relayout.md

@@ -1,9 +1,9 @@
 ---
 type: reference
 title: Media Relayout
-description: "Census, plan and apply the movie library into Jellyfin format: the gates to read, and the rollback."
+description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
 tags: [dankserver, media-server, jellyfin, plex, procedure]
-timestamp: 2026-10-30T00:00:00Z
+timestamp: 2026-11-02T11:00:00Z
 ---
 
 # Media Relayout
@@ -12,17 +12,24 @@ The movie library on [[Entropy Volume]] arrived as release folders —
 `Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/` with the video inside named after
 the release. [[Jellyfin]] matches a movie by folder and file name, so the
 release noise cost posters, extras and versions. These scripts turn that
-library into Jellyfin's own shape — and move TV back out of it — and nothing
-else in the repo touches a media file.
+library into the shape its media server reads — the movies, the TV packs, the TV
+that turns up in the movie share, the music library, and the audiobooks, whose
+only reader is Plex — and nothing else in the repo touches a media file.
 
 | Script | Touches media? | Output |
 |---|---|---|
-| `scripts/media-census.py` | reads headers only | `enum`, `probed`, `plex` TSVs |
+| `scripts/media-census.py` | reads headers and the servers' own DBs, never writes | `enum`, `probed`, and one authority TSV per library |
 | `scripts/media-plan.py` | no — TSVs in, TSVs out | the plan + the exception report |
 | `scripts/media-apply.py` | **renames, never copies**; sweeps only what a run emptied | the moves, plus a journal |
 | `scripts/media-tv.py` | no — TSV out | a plan that moves TV into the TV library |
+| `scripts/media-shows.py` | no — TSV out | a plan that re-files the TV library itself |
+| `scripts/media-music.py` | no — TSV out | a plan that re-files the music library by its tags |
+| `scripts/media-audiobooks.py` | no — TSV out | a plan that re-files the audiobooks by the book Plex matched |
+| `scripts/media-verify.py` | no — reads the servers' DBs read-only | the verdict that every journaled move landed |
 
-The seam is `tests/media_relayout.bats`; the target contract is quoted from
+The seams are `tests/media_relayout.bats`, `tests/media_shows.bats`,
+`tests/media_music.bats`, `tests/media_audiobooks.bats` and `tests/media_verify.bats`;
+the movie and TV target contracts are quoted from
 [jellyfin.org/docs/general/server/media/movies](https://jellyfin.org/docs/general/server/media/movies)
 in `media-plan.py`'s docstring.
 
@@ -40,8 +47,10 @@ $C plex      --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv
 ```
 
 `plex` reads the Plex metadata cache (`--db` defaults to the path the
-[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with
-`mode=ro&immutable=1`: a live server is never locked or written.
+[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with plain
+`mode=ro`: a live server is never locked or written, and plain ro is also what
+makes a scan's uncheckpointed rows visible. The same rule as
+`scripts/media-verify.py` — see *Proving the move landed*.
 
 Then plan — this reads only the three TSVs, so it costs seconds and can run
 anywhere, including a laptop:
@@ -62,47 +71,90 @@ that would be filed wrong, and the plan is not run until it is understood.
 ```bash
 scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies   # dry, writes nothing
 scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
-scripts/media-apply.py verify   --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
-scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies   # dry
-scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
-scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
+scripts/media-apply.py verify   --plan /tmp/jf-plan.tsv --journal /mnt/entropy/.media-campaign/media-relayout.jsonl
+scripts/media-apply.py sweep    --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies   # dry
+scripts/media-apply.py sweep    --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
+scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
 ```
 
+**Every campaign artifact — plan, journal, exceptions file, run log — lives in
+`/mnt/entropy/.media-campaign/`.** The array root carries libraries and nothing
+else: a root strewn with `*.jsonl` and `*-plan.tsv` gets browsed as content, and
+the dot-directory is ignored by both media servers. Relocated 2026-11-02, 48 files,
+undo with `media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-campaign-relocate.jsonl --root /mnt/entropy --apply`.
+The holds and trash (`.AGENTTRASH`, `.HOLD-<date>-*`) stay at the root on purpose:
+the purge machinery is given those paths as `--hold`, and a hold that moves out
+from under the tool that made it is a hold nobody can find. Name `--journal`
+explicitly on every run — the tool's default is `media-relayout.jsonl` beside
+`--root`, so a bare run recreates exactly the root-level log this page forbids.
+
 An apply also leaves the release folders it emptied standing where they were —
 hundreds of them at the library root — which is what `sweep` removes, dry-run
 first, from the same journal.
 
-After it lands, both servers need a scan or the change is invisible.
+`--batch N` (default 64) makes the journal durable per batch instead of per
+move: N intents are written and fsynced, then those N rename. The rule a crash
+relies on — no rename runs before its intent is on disk — is unchanged, so an
+interrupted apply still replays the batch and resumes with nothing lost. Batch 1
+is the old behaviour; the only reason to name it is debugging, because on the
+spinning array it spends most of a large apply's wall time in fsyncs.
+
+After it lands, both servers need a scan or the change is invisible. The two
+drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
+and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
+with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
+them in that order and never together: Jellyfin's only door walks every library,
+every Plex section forks its own scanner walk, and the array is the bottleneck
+both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
+of them decide this step: `204` is acceptance and anything else queued nothing —
+re-issue the identical request rather than diagnosing — and ONE accepted refresh is
+the whole procedure, because a second while a scan runs moves `Scan Media Library`
+to `Cancelling` and every later `204` is accepted and then discarded.
+
+### Proving the move landed
+
+A scan answers `204` whether or not it finds anything, so proof is the journal
+replayed against both servers' own tables: every `move` row has a row at its
+`dst`, and **zero** rows still name its `src`. `scripts/media-verify.py` is that
+replay: it opens each database read-only (there is no `sqlite3` binary on these
+hosts or inside either container) and exits non-zero while a row is stale.
+
+Both libraries live in WAL mode, so the tool opens them plain `mode=ro`:
+`immutable=1` promises sqlite the file never changes, which skips the
+write-ahead log — a scanner's own uncheckpointed rows then read as missing, and
+a read beside a running scan can fail as "database disk image is malformed".
+`immutable` stays only the fallback for a database whose `-shm` cannot be read.
 
 ```bash
-# Run this from the host shell through `podman exec`, never from a neighbour
-# container: Plex's HTTPD closes without a single byte on any request that
-# arrives from another container's netns (10.89.x.x reads as "empty reply", not
-# as a 401). The token rides stdin, never argv, where any process on the host
-# could read it out of /proc/<pid>/cmdline.
-umask 077
-podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
-  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
-podman exec -i plex sh -c 'read -r T
-  for k in $(curl -s -H "X-Plex-Token: $T" -H "Accept: application/json" \
-      http://127.0.0.1:32400/library/sections | grep -o "\"key\":\"[0-9]*\"" | cut -d\" -f4); do
-    curl -s -o /dev/null -w "plex section $k: %{http_code}\n" -X POST \
-      -H "X-Plex-Token: $T" "http://127.0.0.1:32400/library/sections/$k/refresh"
-  done' </tmp/px.tok
+scripts/media-verify.py --journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl \
+    --db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
 ```
 
-[[Jellyfin]] has no credential on this box that can trigger a scan: the two keys
-in its database are Seerr's, unused since 2026-09-01, and answer 401 to every
-call. Mint one (*Dashboard → Advanced → API Keys*, Administrator) and store it in
-`~/.config/secrets/jellyfin`, or press *Library → Scan All Libraries* in the web
-UI. A scripted scan passes the key in a mode-600 curl config, never in a header
-argument:
+"No row at the destination" is not one verdict but four. **never indexed**: the
+server indexes no file of that extension (its own extension census says so) — a
+sidecar, not a lost film. **gone**: the file sits at neither src nor dst, because
+a later pass or the server's own extension sweep renamed it. **phantom**: a row
+still naming src while the file has left src. **REAL**: no row at dst *and* the
+file still at src — the only class that is this relayout's own damage. Phantoms
+are normal for both servers (Plex carries thousands that predate any relayout);
+re-scan to clear them and never read them as loss.
 
-```bash
-umask 077; printf 'header = "X-Emby-Token: %s"\n' "$(cat ~/.config/secrets/jellyfin)" >/tmp/jf.hdr
-curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" \
-  -X POST http://127.0.0.1:8096/Library/Refresh
-```
+`media-verify.py` prints this split itself — `on disk at dst, no row names it`,
+`REAL — file still at src`, `gone — at neither path` — so nobody hand-stats a
+thousand paths to find out whether a film is lost or merely unindexed.
+
+The owning server decides whether a pass can convict anything: Plex's roots are
+its `sections` rows joined to their location rows (`Movies`, `TVShows`, `Music`,
+`Audiobooks`, `LiveRecording`) and Jellyfin holds no audiobook library at all, so
+the audiobook journal verifies with `--db-kind plex` or it proves nothing.
+
+Three doors that look right and are not: `GET /search?query=` matches metadata
+titles, never file paths; `?path=` on `/library/sections/N/all` makes Plex's
+HTTPD close without a single byte; and the same HTTPD drops requests that carry
+no `Accept` and client-identifier header (see the scan block above for the form
+that works).
+
+A duplicate prune is the same census and the same proof, with its own keep rules — films by title and duration, a series per episode: [[Media Duplicate Prune]].
 
 ## TV in the movie library
 
@@ -114,11 +166,11 @@ by series, and writes a plan in the executor's own format:
 
 ```bash
 scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
-    --out /mnt/entropy/media-tv-plan.tsv
-scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
-scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
-    --journal /mnt/entropy/media-relayout-tv.jsonl --apply
-scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
+    --out /mnt/entropy/.media-campaign/media-tv-plan.tsv
+scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy
+scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy \
+    --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
+scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl \
     --root /mnt/entropy --apply
 ```
 
@@ -133,13 +185,35 @@ from the box-set folder. What it refuses it prints and leaves alone:
 - a destination two files both claim: the executor refuses the whole plan
   before its first rename, and only Frank decides which copy is the keeper.
 - an episode the destination library already holds at the same byte count. The
-  movie share carried a second copy of all 124 Monk episodes — the TV library
-  has had the same box set for years, every episode the same size, ~31 GiB of
-  it. The mover leaves such a series in place and says so; a duplicate is
-  deleted or kept by Frank, never copied into one library by a tool.
+  movie share once carried a second copy of a whole box set — same episode list,
+  every byte count identical. The mover leaves such a series in place and says
+  so; a duplicate is deleted or kept by Frank, never copied into one library by
+  a tool.
 
 A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
 
+Below the threshold the mover prints and leaves, so a lone downloaded episode of
+a real show stays in the movie library on purpose.
+Filing one by hand is the same executor on a hand-written plan and the same
+journal; the trap is the series year. Read it from the file's own name or the
+folder it owns, never from the title whose bonus folder it happens to be parked
+in — `The Cleveland Show S03E07` inside `Die Hard (1988)/extras/` is not a 1988
+show. A name whose marker is `Part 2` is a film and stays put.
+
+## The TV library's own relayout
+
+`/mnt/entropy/TVShows` is its own arm: a different authority (tvdb), its own
+refusals, and its own test for whether a folder really holds two shows —
+[[Media TV Relayout]]. It shares this page's executor, the one-journal-per-arm
+law, the sweep and the rollback.
+
+## The music and audiobook arms
+
+Each is a page of its own, because each reads a different authority and refuses
+for a different reason: [[Media Music Relayout]] files by Jellyfin's track tags,
+[[Media Audiobook Relayout]] by Plex's chapter rows. Both share this page's
+executor, the one-journal-per-arm law, the sweep and the rollback.
+
 ## Laws
 
 - **A moved file is a 404 in [[Plex]] until its section is rescanned.** The
@@ -166,6 +240,19 @@ A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
   folder holding it, and extras drifted into a neighbour. A bare number counts
   only when it could be a release year, so `1080` and `2160` never pose as
   agreement between two names.
+- **Compare two media libraries with the series engine, never a path
+  substring.** Filtering paths for `monk` also returns *Monkey Dust*, and
+  letting whichever file a walk happens to reach first stand for an episode
+  paired two libraries into a fake duplicate — the same sloppiness pointed the
+  other way deletes a file that is unique. The comparison is `media-titles.ep_of`
+  on both sides: series, season, episode, bytes, and a hash before anything is
+  trashed.
+- **A deletion is staged on the item's own filesystem.** The trash rule names
+  `~/.AGENTTRASH`, but `/home` and `/mnt/entropy` are different devices: a 31 GiB
+  stage there is a copy rather than a rename, it has to fit in the 187 GiB of
+  free space, and `rename(2)` refuses it anyway. Trash goes beside the item —
+  `/mnt/entropy/.AGENTTRASH/<date>-<topic>/` — so it costs one inode change and
+  rolls back exactly like the move that made it.
 - **A file whose name carries no words is bonus material, not a folder.**
   `.Extra 1 ( 1H 27min multisub ).mkv` parses to `1`; a movie folder named `1`
   is not a movie. It becomes a review line telling a human to name the file.
@@ -180,9 +267,12 @@ A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
 - **The journal lives BESIDE the library root, never inside it** (default:
   `media-relayout.jsonl` next to the root): a log inside the library is a file
   Jellyfin indexes.
-- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
-  duplicate destination stops the run *before the first rename*, so the operator
-  fixes one thing and re-runs instead of untangling a half-applied plan.
+- **Refusals are all-or-nothing.** An occupied target, a vanished source, a
+  duplicate destination, or a plan row whose verb is not `mkdir`, `keep` or
+  `move`, stops the run *before the first rename*, so the operator fixes one
+  thing and re-runs instead of untangling a half-applied plan. An unread verb is
+  refused by line number: silently dropping one turned a pass meant to rename
+  two series folders into `applied: 0 moved` with exit 0 (2026-11-02).
   `--force` relaxes only the opinion guards; it never permits overwriting.
 - **A sweep deletes only what the journal proves the run emptied.** A folder is
   eligible when a journalled file moved out of it or out of something under it,
@@ -194,22 +284,59 @@ A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
   recreates the folder, so the sweep is undone by the same command as the moves.
   Proof carries downward: inside a folder the journal proves emptied, every
   empty directory goes with it. A release's `Screens/` or `Sample/` is never a
-  move source itself, so 146 of them held their emptied parents on disk after
-  the first sweep.
+  move source itself, so it can hold an emptied parent until the next sweep.
   A failed `rmdir` is not a removal: only a folder that is really gone leaves
   scope, so the next sweep finishes what a permission problem stopped.
 - **An episode-shaped name marks a SERIES, not a film — and the series is the
   name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
-  `Monk`. Keyed on the whole title instead, every episode became a group of one
-  and never reached the five-episode size that calls something a series, so a
-  box set was filed as 124 movies in 124 folders. The threshold still earns its
-  keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
+  `Monk`. Keyed on the whole title instead, every episode became a group of one,
+  never reaching the five-episode size that calls something a series — a whole
+  box set filed as one folder per episode. The threshold still earns its keep:
+  a lone film named `Part 2` stays a film. Episodes of a series are parked
   in `_TV needs a TV library - <series>` for a human to move to a TV source —
   the movie tool never files TV.
 - **One journal is one undo.** The TV move runs on its own journal, and with
   `--root` set to the share that holds BOTH libraries: the executor refuses any
   path outside `--root`, and a journal shared with the movie relayout would
   undo both runs when only one of them is wanted.
+- **A hold folder outside `--root` forces `--force` on BOTH `apply` and
+  `sweep`.** `sweep` refuses the entire journal when any journalled path sits
+  outside the root (`this journal belongs to another library`) — the same guard
+  the plan hits, and it exists to stop one pass unwinding another's files.
+  `--force` relaxes it and stays safe there: `sweep_targets` marks only
+  ancestors under `--root`, so a hold folder beyond the root can never be
+  selected; it only blocks everything that is. The cleaner shape is the law
+  above — put `--root` on the share that holds both.
+
+- **Sidecar metadata does not follow a rename, because it lives on another
+  filesystem.** The media servers write `.nfo` and artwork through the
+  [[Media Metadata View]] (NVMe upper layer); this tool moves bytes on the raw
+  array path, so where that view is mounted a sweep or a move must be followed by
+  `scripts/media-view.sh reap --apply` — stranded sidecars become ghost folders
+  that both servers browse and report as content. Whether this host mounts the
+  view at all is stated on that page; `media-apply.py` stays silent when it does not.
+  A rename of folders the server never identified strands nothing: Jellyfin
+  writes a sidecar only for a series it recognised, so `reap` answering
+  `no ghosts` after such a pass is the expected result, not a broken tool.
+- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
+  whiteout in the upper layer, which hides the array's copy without removing it;
+  the file returns when the view is unmounted. Deletions in this pipeline happen
+  on the raw path only, which is what the journal can undo.
+- **The music album key is case-folded; the folder name is not.** `Master Of
+  Puppets` and `Master of Puppets` are one album to the matcher, so the spelling
+  that reaches the folder is the one most tracks carry. Taking it from whichever
+  row sorted first made the folder name depend on row order, and re-planning an
+  already-filed library renamed albums that were already in place.
+- **An occupied path is never a destination**, not even when its occupant is
+  scheduled to leave: the executor refuses an occupied target, so a plan that
+  aims at one fails after the renames that were supposed to clear it. A file
+  standing where the contract puts it is a keep, and a file already carrying a
+  collision suffix keeps that suffix — otherwise the plan is a shuffle, not a
+  fixed point, and a re-run moves albums that are already right.
+- **`media-census.py music` reads the `BaseItemProviders` table, not the item's
+  JSON.** This Jellyfin build carries an empty `ProviderIds` inside
+  `BaseItems.Data` for albums, so a census that parses the JSON finds no
+  MusicBrainz or Spotify id on any album and reads as "the library is untagged".
 
 ## Rollback
 
@@ -219,9 +346,17 @@ any folder a sweep removed, restores every source path, removes only the folders
 that run created *and that are now empty*, and refuses everything if any moved
 file cannot be accounted for. A rolled-back journal refuses to roll back twice.
 
+**Roll a chain back newest journal first, or the tool refuses.** Each pass
+re-files what an earlier one moved, so a journal is a clean inverse only until
+something downstream touches its files; `ls -l /mnt/entropy/media-relayout*.jsonl`
+is the order. Refusing is the tool working: on 2026-10-31 the movie journal could
+not restore files the later TV passes had re-filed, and the stray journal could
+not restore files Jellyfin's extension sweep had renamed. Roll back what still
+inverts and leave the rest as the current state — never force it.
+
 ## See Also
 
-[[Jellyfin]], [[Plex]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
+[[Jellyfin]], [[Plex]], [[Media Metadata View]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
 
 ## Citations
 

serverAppConfigs-docs/Dankserver/Media TV Relayout.md

@@ -0,0 +1,241 @@
+---
+type: reference
+title: Media TV Relayout
+description: "Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test."
+tags: [dankserver, media-server, jellyfin, plex, procedure]
+timestamp: 2026-11-02T11:00:00Z
+---
+
+# Media TV Relayout
+
+The `/mnt/entropy/TVShows` share arrived as scene packs — release folders whose
+names are the release, not the show. This arm re-files it into the shape
+[[Jellyfin]] reads, with tvdb as the name authority. The executor, the journal,
+the sweep and the rollback belong to [[Media Relayout]]; this page owns the TV
+authority order, the refusals, and the test that decides a split.
+
+`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
+already matched and numbered most of it. `scripts/media-shows.py` re-files that
+library into the shape Jellyfin reads, with **tvdb as the name authority**:
+
+    <Series> (<year>) [tvdbid-NNNN]/Season NN/
+        <Series> (<year>) - SxxEyy - <Episode Title>.ext
+
+It never guesses: every series name, episode number and title it writes traces to
+an authority, and what has none is parked for a human to read.
+
+```bash
+# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
+#    and Tvdb ids and the title the row holds (the DB opens read-only).
+python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
+    --out /tmp/tv_titles.tsv
+# 2. What is on disk (stat only), and the runtimes the film arm needs.
+python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
+    --out /tmp/tv_enum.tsv
+python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
+    --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
+#    The probe is the slow step: ffprobe reads both ends of every file. So re-use
+#    the last one — a duration belongs to the file, not to its name: re-key the
+#    previous census through that pass's own journal (`src` -> `dst`) instead of
+#    seeking the array again. It is also the only column the film arm reads, and
+#    only for a file with no marker and no provider row (4,794 of 58,551 today,
+#    all of them covered by a re-key that costs seconds, not an hour).
+# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
+#    which travels as the `api_key` query parameter. Only a v4 request token is
+#    accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
+#    which the summary line flags as NOT provider-complete. --tvdb reads its
+#    `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
+#    cache file, so the plan is reproducible offline afterwards: run it once to
+#    fill the cache, and quote the second run.
+#    Paths are only strings to a dry run, so this step can run on the workstation
+#    against scp'd TSVs — which is where the two provider secrets live.
+python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
+    --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
+    --tmdb-cache /mnt/entropy/.media-campaign/media-tv-names.json --tmdb --tvdb \
+    --park /mnt/entropy/NeedsProcessing/TVShows \
+    --out /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
+    --exceptions /mnt/entropy/.media-campaign/media-relayout-shows2-exceptions.txt
+# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
+#    as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
+#    applied pass is its unwind record, so a second pass that reuses the name
+#    destroys the only map back.
+scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
+    --root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-shows2.jsonl --apply
+```
+
+Evidence, strongest first — the order the planner believes things:
+
+- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
+  run longer than three at that position is NOT a number — `S03E2019` and
+  `S04E47370355` are one box set's airdate fragment and pack index — so such a file
+  is renumbered by tvdb's title or parks, and is never truncated down to fit;
+- the **`Season NN` folder it sits in and the number its name leads with inside
+  it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
+- the **provider row**: which episode the file IS, and the numbering of a file
+  whose own name says nothing;
+- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
+  an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
+  episode title, because a name the server read off a folder is not a show name.
+  The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
+  Plex's scanner both read, so a show whose title is ambiguous resolves by number
+  instead of by a human guessing which folder is which. The id rides the folder
+  only — a file name gets 180 characters and the episode title needs the room.
+- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
+  series' canonical extra folder;
+- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
+  when the movie provider's title is a run of words the FILE itself states and its
+  year agrees with the file's: the query is only what `title_of` salvaged from a
+  release name, and the first hit for a short one is a near miss — a 1963 Bond film
+  for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
+  parks.
+
+Four refusals carry the design: what the planner will NOT do is the point of it.
+
+- **Folders resolve a series; they never invent one.** A folder's words may
+  resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
+  Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
+  Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
+  `…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
+- **A scraped row does not outrank the filesystem.** Jellyfin numbers
+  multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
+  season 16 episode 20, and one Gundam row says season 2004. The folder, then
+  the file's own leading number, win.
+- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
+  specials by airdate, so a row can arrive saying `season 2002`; that is not a
+  season and it never becomes a folder. The one authority on such a row is tvdb's
+  episode list read backwards: the title the row carries names the (season,
+  episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
+  the file parks with the reason it always carried. `Season 00` IS a legal
+  season — specials live there — so a row that really says 0 files there instead
+  of parking.
+- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
+  numbered by its release group books `S05E01` in a season tvdb does not hold, or
+  books more episodes there than tvdb lists: the slot its number points at is a
+  different episode, and dressing it in that episode's title is a confident lie.
+  The file still renames to the standard shape; the title does not ride, and the
+  series is listed under *seasons whose numbering tvdb contradicts* in the
+  exceptions. A season holding FEWER files than tvdb lists stays plausible — a
+  partial pack is the normal shape. Neither reading holds for a list tvdb
+  cut off at its own page limit: the default episode query answers at most
+  500 rows and states no total, and One Piece, Crayon Shin-chan and
+  Detective Conan each answer exactly 500 while holding over a thousand —
+  a page bounds nothing above itself, so it contradicts nothing.
+
+**A provider name is evidence only when an id backs it.** Jellyfin names a Series
+after the folder it scanned, so a library filed by release folders ends up with
+one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
+copying those names verbatim launders folder junk into the contract. The planner
+resolves each series to its tvdb id and writes tvdb's name and year; the merged
+identity is also what turns a box set split across five release folders into ONE
+folder, and what makes two files claiming one episode MEET in the duplicate rule.
+A series no id resolves keeps the provider's words and is listed under *no tvdb
+identity* in the exceptions — that list, not a rename, is the review.
+
+**The review pile lives outside the library.** Files parked inside
+`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
+show's rows then looked like evidence. So a row whose series is the hidden
+top-level folder the file itself sits in is dropped — and the folder is matched
+with its leading underscore stripped, because the server *displays* `_unsorted` as
+`unsorted`: compare the spelled names and the pile survives the relayout as a
+visible series folder that real files get renamed into. `--park` outside the root
+makes the next plan re-file the pile instead of skipping it. Inside a hidden
+top-level folder a file's own words may MATCH a registered show, never invent one.
+
+**The operator's id map is the door for a name tvdb cannot be searched by.**
+`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
+line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
+tvdb titles a record in the show's own language and the English name rides in
+*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
+spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
+2023 live action, not to the 1999 record (81797) the pack here is. The map
+outranks a provider id — a box set's provider id is the server's own guess at a
+folder name — and it merges the release folders that are one show. Everything it
+touched is written to the exceptions: every folder it named, every row that
+matched no folder, every disagreement with a provider id, every id tvdb never
+answered. A line whose id is not a number stops the plan, because a typo renames
+a whole pack to the wrong show. Only folders the search CANNOT reach belong in
+it; a row for a folder the search would resolve hides a live lookup behind a
+number nobody re-checks.
+
+`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
+plan's `mkdir` rows, asks tvdb about every destination folder carrying no
+`[tvdbid-`, and writes one row per folder with the verdict that says how much was
+guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
+release junk is stripped from the folder's words), `NEAR` (tvdb answered
+something else — the best hit is printed for a human and nothing applies it),
+`NONE`. Acceptance never moved across the passes that built this: a hit counts
+only when tvdb's name equals the query with every non-alphanumeric dropped from
+both sides, and the record's year still has to fall inside any year span the
+folder names. That gate is what caught three confident picks — Fairy Tail's 2024
+sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
+Clover` (378762) offered for Black Clover. It is why the map is approved by hand
+and the sheet is never fed to the planner.
+
+**A parked folder merges into the row that owns the show.** The folder that never
+got an id usually holds episodes the identified folder for that show lacks, and
+the server shows the audience neither: the parked copy is invisible and the owner
+reads as a short series. One `media-apply.py` plan per show, built from the
+server's own rows:
+
+- the **owner is the folder whose `Series` row carries the tvdb id**; the parked
+  side is the folder with the same words and no id (or the same id under a second
+  row — see the law below);
+- the **comparison is episode identity, not filename**: which (season, episode)
+  the provider row gives each file, read straight out of the server. Release
+  folders renumber — one park carried `S01E01-11` of a show tvdb books as seasons
+  2 and 3, and every episode runs the same eleven minutes, so neither the marker
+  nor the duration says what one query into the rows says;
+- **a season the owner has no file in moves whole**: `mkdir` the `Season NN`,
+  then one row per file with its name unchanged — the folder shape is the
+  contract, the file name is not;
+- **a subtitle pack is re-keyed, never just moved**: a sidecar attaches only when
+  its stem equals the video's, so each `.srt` has to land on the OWNER's stem with
+  the language suffix kept (`… - The Cat and the Claw (1).en.srt`). Moved under
+  its release-group name it stays a folder of 88 subtitles no player ever offers;
+- **the same episode on both sides is the operator's call**, never a plan row:
+  which copy survives is a quality judgment — the parked copy is usually the
+  bigger one, and a re-cut is not a worse one — and the parked file often carries
+  no title to align by. Those pairs are listed and left alone.
+
+The two sides speak different path dialects, since the server records the
+container coordinate ([[Jellyfin]]) and the journal speaks the array; a join that
+forgets the translation returns zero rows and looks like an empty library.
+
+## Laws of this arm
+
+- **A folder that names two series is not evidence of two series — the files
+  answer.** `ffprobe -show_entries format_tag=title` names the series each file
+  belongs to. Measured 2026-11-02 on `COW and CHICKEN … Complete Animated TV
+  Series`, a bundle whose name pairs *Cow and Chicken* with *I Am Weasel*: 103
+  of 104 files carry a `Cow and Chicken - SxxEyy - <title>` tag and none carries
+  an *I Am Weasel* one. The 7-minute duration class is a half-episode segment,
+  not a second show, and a file holding both segments needs a remux, which this
+  executor will never do. Split only when the tags (or a provider episode list)
+  name both series per file; everything else is a rename.
+- **When the numbering is right and only the name is wrong, rename the folder
+  and nothing else.** Tags whose `Sxx` matches the `Season NN` folder holding
+  them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove
+  the layout is already correct. Renumbering files under that evidence buys
+  nothing and can silently mislabel an episode.
+- **One tvdb id can carry two `Series` rows, so the id-less folder list proves
+  nothing about a merge.** A parked folder whose words parse to a registered show
+  gets identified on its own and keeps its own folder: the count of id-less
+  folders drops, and the library now offers the same show twice. Prove a merge by
+  the episode rows standing under the owner folder, never by that list.
+- **Move library files only while `Scan Media Library` is Idle.** A rename that
+  lands under a running scan is a row the server read a moment ago pointing at a
+  path that no longer exists. `ScheduledTasks` is the read ([[Jellyfin]]); wait
+  for a `Running` sighting and then two consecutive `Idle` polls, because a gap
+  between sub-tasks reads as finished.
+
+## See Also
+
+[[Media Relayout]] — the executor, the journal, the sweep and the rollback this
+arm shares. [[Jellyfin]] — the reader of the resulting shape. [[Plex]] — the
+second reader. [[Media Metadata View]] — where a rename's sidecars strand, or
+do not.
+
+## Citations
+
+- Jellyfin Shows documentation — series, season and episode folder/file naming:
+  <https://jellyfin.org/docs/general/server/media/shows>

serverAppConfigs-docs/Dankserver/Plex.md

@@ -4,7 +4,7 @@ description: "Plex Media Server is a **containerized media server on Dankserver*
 tags:
   - dankserver
   - plex
-timestamp: 2026-10-21
+timestamp: 2026-10-30
 ---
 
 # Plex
@@ -13,7 +13,8 @@ Plex Media Server is a **containerized media server on Dankserver**: a podman
 quadlet (`plex.service`) under the systemd **user** generator, generated from
 `dankserver/plex/quadlet/plex.container`. It shares the host with a second live
 media server, [[Jellyfin]] (`jellyfin.service`, listening on 8096); both serve
-from the same read-only Entropy Volume mount.
+from the same tree — the [[Media Metadata View]] overlay — so the Kodi-format
+sidecars Jellyfin writes are the files Plex reads, with no copy and no sync.
 
 **Host:** Dankserver — **Ports:** 32400/tcp (PMS web + API), 32400 is the
 only port PMS needs for client streaming; LAN clients connect directly,
@@ -26,7 +27,7 @@ public traffic can be fronted by Caddy like every other vhost.
 | Runtime | podman quadlet, `plex.service` (user manager) — migrated from the native `plexmediaserver.service` OS-package unit (2026-09) |
 | Image | `lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329` (pinned; verified == the registry's `latest` manifest digest at pin time: `sha256:8f655bad…e9e2334`) |
 | Data root | `/var/lib/plexmediaserver` bind-mounted **in place** as `/config` (RW) — the migration moved zero bytes |
-| Media | `/mnt/entropy` mounted at the same absolute path **read-only** — every library path in the PMS database stayed valid, zero re-scan |
+| Media | `/var/lib/media-meta/view` (the [[Media Metadata View]] overlay) bound at the same absolute path `/mnt/entropy` **read-only** — live since the 2026-10-31 cutover, so the PMS database's recorded paths keep resolving with zero re-scan |
 | Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
 | Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
 | Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
@@ -38,7 +39,9 @@ public traffic can be fronted by Caddy like every other vhost.
 
 The migration moved zero bytes: the container bind-mounts the native data
 root where it already lived, and the media archive re-mounts at
-`/mnt/entropy` inside the container, so every absolute library path in the
+`/mnt/entropy` inside the container — only the HOST side of that bind has moved
+(the array until the 2026-10-31 view cutover, the overlay since), so every
+absolute library path in the
 PMS database keeps resolving. The one metadata change: rootless podman
 cannot map the real host uid 997, so the fleet model applies — container
 root runs as the `dank` user, and the deployer adopts the data root with
@@ -83,4 +86,78 @@ root runs as the `dank` user, and the deployer adopts the data root with
 4. **Cold container boot is slow.** PMS pre-binds 32400 and empty-replies
    while the HTTP layer finishes loading; the first boot of a years-old
    library took ~10 minutes (2026-09 cutover). The deployer gate now allows
-   600 s before failing; later boots are much faster (warm page cache).
+   600 s before failing; later boots are much faster (warm page cache). A
+   `systemctl --user start` issued after an `unmask` is a job in the manager,
+   not in the shell that issued it: it lands minutes later and survives that
+   client dying, so a wrapper script that looks hung is usually only waiting —
+   read `is-active` and `podman ps` before declaring a fault, and give every
+   step of such a script its own `timeout`.
+5. **`32400` answers TLS only to remote clients.** PMS requires secure
+   connections, so `curl http://192.168.0.235:32400/identity` from any other
+   machine closes without a single byte (curl exit 52) — that reads as a dead
+   server beside a healthy one, and it is how a post-cutover health check gets
+   misread. Probe with `curl -k https://192.168.0.235:32400/identity`. Two
+   consequences: `plex.franksdank.cloud` works **only** because its `transport
+   http` block carries `tls_insecure_skip_verify` (that option makes Caddy speak
+   TLS to PMS's self-signed cert — delete it and the vhost 502s), and the scan
+   block below rides `http://127.0.0.1` **inside** the container, where insecure
+   connections are allowed.
+
+## Triggering a scan
+
+A rename on disk is invisible until the section is rescanned, and a moved file is
+a `Couldn't find the file to stream` 404 until that scan passes it. Run the
+driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
+`5` Audiobooks, `6` Music, `11` VODs & Live Events:
+
+```bash
+scripts/media-refresh-plex.sh trigger --library 1
+python3 scripts/media-verify.py --db-kind plex \
+    --journal /mnt/entropy/.media-campaign/<journal>.jsonl   # the real proof
+```
+
+Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
+five for a rename that touched one buys nothing. `trigger` answers
+`refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
+still warming, which the driver retries out loud; a service restart is never the
+fix.
+
+**`status` is a census, not a waiter: no epoch makes it watch a scan.** It reads
+`section_locations.scanned_at` — the epoch of the last scan PMS has *recorded* for the
+section, the same value `GET /library/sections` prints as `scannedAt` (verified
+2026-11-04) — and compares it against `--after`. Measured twice on 2026-11-04: a chain
+fell through its wait loop in the same second its own trigger was accepted, and a direct
+read 30 minutes later still answered `settled` off that one stamp
+(`2026-11-04T19:46:59Z`, for a request accepted at `19:46:58Z`) while the walk ran. A
+journal's mtime fails, and so does the trigger's own epoch — the request satisfies the
+very door it was asked to wait for. The failure direction is a false *success*, never a
+false alarm, which is what makes it worth the warning: ask `status` whether a scan got
+recorded, and never let a chain depend on it.
+
+**The proof that a scan did its work is the journal replayed against the database:**
+`scripts/media-verify.py --db-kind plex --journal <journal>` names every row that still
+points at a source path (`still at src`) and every entry whose file is physically still
+there (`REAL`). Both at zero is what gates a purge, and it needs no settle signal at all
+— a scan can clear rows but cannot invent one. Jellyfin's door has no such trap:
+`scripts/media-refresh-jellyfin.py wait` watches the scan task itself and refuses by
+name when no task reaches `Running`.
+
+The driver ships its runner into the container with `podman cp` rather than
+quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
+a single byte on a request that arrives from another netns — the host address and
+the `10.89.x.x` container address both read as "empty reply", never as a 401 —
+and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
+error long before it reaches the server. The request therefore rides plain
+`http://127.0.0.1:32400` from inside the container, where insecure connections
+are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
+`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
+read inside the container into a mode-600 curl config there, so it never enters
+argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
+URLs). Hand-writing that config has one more trap: inside a quoted `printf`
+format a backslash-newline is LITERAL, so a two-line string becomes one glued
+line plus a stray backslash, and curl answers `option -K: is unknown` — never
+"bad config line".
+
+Proof that a rename landed is not a scan response: `scripts/media-verify.py`
+replays the run's journal against the database — the only read that
+distinguishes "not scanned yet" from "gone"; see [[Media Relayout]].

serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md

@@ -4,7 +4,7 @@ description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for t
 tags:
   - infrastructure
   - authentik
-timestamp: 2026-10-19
+timestamp: 2026-11-03
 ---
 
 # Authentik Outpost Reference
@@ -56,7 +56,7 @@ correctly — the outpost matches on hostname.
 
 ## Provisioning rules for proxy providers
 
-Two provisioning rules, each verified live by an outage:
+Three provisioning rules, each verified live by an outage or a dead login:
 
 1. **All three flows are required.** A ProxyProvider created via raw API/ORM
    with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
@@ -80,6 +80,16 @@ Two provisioning rules, each verified live by an outage:
    the app. No error is logged anywhere — the SSO "works" and the app never
    appears. Create the Application with `provider_id` set (the UI does this
    automatically; raw API/ORM drives must do it explicitly).
+3. **A raw ORM create skips the OAuth defaults.** The REST serializer calls
+   `ProxyProvider.set_oauth_defaults()`; `ProxyProvider.objects.create()` does not. The row
+   then looks fully configured — external_host, internal_host, application bound, outpost
+   claimed — while `redirect_uris` is empty (those URIs are *derived* from `external_host` by
+   `_get_callback_url`) and `grant_types` is `[]`. Both kill the guard before any credential,
+   in this order: authentik renders its own **"Redirect URI Error"** page, and once the URIs
+   are filled by hand the next wall is a bodiless **HTTP 400** out of `/application/o/authorize/`
+   (both measured on the Audiobookshelf guard provider, 2026-11-03). Census and repair in one idempotent pass, all
+   providers at once: `scripts/authentik-proxy-provider-defaults.sh`; `--check` prints the
+   census and exits 1 on drift, and a provider that already matches is left alone.
 
 ## OAuth2 relying-party quirks (this fork; corrected live 2026-10-20)
 
@@ -138,36 +148,46 @@ Read from the live database (read-only `SELECT`s against the
 | Object | Value |
 |---|---|
 | LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
-| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
-| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
-| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
+| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` — **verified by a live bind 2026-11-04** (the bind and the `memberOf` search both succeed, so the `ou=users` path is right) |
+| Service account | `svc-jellyfin-ldap` (user pk 8). It has a usable password and authenticates through the default flow — that password is held by Jellyfin in the plugin config (`LdapBindPassword`) and is what the bind actually uses (live bind 2026-11-04) |
+| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — and **unused by Jellyfin**, which has no token field (see the rule below) |
 | RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
 | Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
 | Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
 
-Hard-won rules (the first is now contested by the live database):
+Hard-won rules:
 
-- **Token intent: contested, do not trust either value blindly.** An earlier
-  revision asserted that a bind token with `intent=app_password` is rejected by
-  the flow's password stage with `Invalid password` and only
-  `intent=verification` passes. The live token for `svc-jellyfin-ldap` is
-  `app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
-  wrong or the current Jellyfin bind path is not that token. Nothing observable
-  settles it today: the LDAP outpost's retained log holds only its 5-minute
-  `initialized direct binder` / `Update providers` config cycles and no bind
-  activity *(unverified 2026-10-18: no bind was performed — reproducing it
-  requires the bind secret)*. Re-verify with a real bind before changing the
-  token's intent either way.
+- **Token intent is moot for Jellyfin.** The earlier argument (a bind token with
+  `intent=app_password` is rejected by the flow's password stage, only
+  `intent=verification` passes) never applied here: LDAP-Auth sends the service
+  account's **password** from `LdapBindPassword`, not a token. A live bind plus a
+  `memberOf=cn=jellyfin_users` search (11 entries) succeeded from Dankserver on
+  2026-11-04. Leave the token row alone.
 - **Search needs `search_full_directory`.** Without the RBAC grant the
   outpost returns only the bind user itself — a `(memberOf=…)` filter then
   matches zero entries and the plugin logs `Found no users matching X`.
   The permission is object-level on the provider; grant it via an RBAC role
   and retest.
+- **The outpost log never shows a bind.** Its retained journal is only the
+  5-minute `initialized direct binder` / `Update providers` config cycle; the
+  absence of a bind line is not evidence of anything. Instruments that do answer:
+  `ldapsearch` with the plugin's own DN, password and filter, and the authentik
+  event registry (`login` / `login_failed` with `client_ip` from the podman
+  gateway, since the published port SNATs).
 - `ak shell` import path for the provider model in 2026.8:
-  `from authentik.providers.ldap.models import LDAPProvider`.
-  - (unverified 2026-10-18: importing it outside a configured Django shell fails
-  with `ImproperlyConfigured`, and no `ak shell` was driven against the live
-  auth service for this check)*
+  `from authentik.providers.ldap.models import LDAPProvider` (verified inside a
+  live `ak shell`, 2026-11-04 — it only fails outside a configured Django shell).
+- **`ak shell` field names, 2026.8:** `Event` is keyed by `created` and `action`
+  — there is no `timestamp` and no `type`; `Outpost` has `providers` but **no
+  `apps`**. Introspect first: `[f.name for f in Model._meta.get_fields()]`.
+- **Never pipe `ak shell` inside a heredoc.** `podman exec -i … ak shell | grep
+  -v '^{'` binds the heredoc to `grep` (the redirection belongs to the LAST
+  element of the pipeline), so the shell gets no stdin and prints nothing at all.
+  And `2>/dev/null` on `ak shell` eats the traceback, which looks identical to "no
+  rows". Clean pattern: have the script write its results to a file and read that
+  file back —
+  `podman exec -i authentik-server ak shell >/dev/null 2>/dev/null <<'PY' … open("/tmp/x.txt","w") … PY`
+  then `podman exec authentik-server cat /tmp/x.txt`.
 
 ## Self-serve user-settings flow (password change)
 

serverAppConfigs-docs/Infrastructure/Authentik.md

@@ -145,18 +145,17 @@ relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
 The second outpost — `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
 → `authentik-outpost-ldap.service`, one of the five units in this stack —
 serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`
-`Restart=always`, container up, 2026-10-18). The image is pinned by its **amd64
-single-platform digest**
-`ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9`
-(`dankclaw/authentik/quadlet/authentik-outpost-ldap.container:26`, matching the
-running image in `podman ps`, 2026-10-18). The digest in this page before 2026-10-18
-(`…fa33bc8f…`) is superseded. The reason for pinning a digest at all: podman
-5.7.0 refuses the plain tag because it resolves to a *nested* manifest list
-(index → per-platform manifest lists) — `unexpectedly received a manifest list
-instead of a manifest for a single image` — first hit on `:2026.8.0`, still the
-rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
-<image> | jq -r '.manifests[] | select(.platform.architecture=="amd64") |
-.digest'` and pin that digest.
+`Restart=always`, container up, re-checked 2026-11-04). **The image pin has one
+owner: the `Image=` line of
+`dankclaw/authentik/quadlet/authentik-outpost-ldap.container`** — never restate
+the digest here, because every pin bump turns this page into a lie (the digest
+cited here went stale twice: `…fa33bc8f…`, then `…2e7b259e…`). The reason a
+digest is pinned at all: podman 5.7.0 refuses the plain tag because it resolves
+to a *nested* manifest list (index → per-platform manifest lists) —
+`unexpectedly received a manifest list instead of a manifest for a single image`
+— first hit on `:2026.8.0` and still the rule. Resolve once with
+`podman manifest inspect <image> | jq -r '.manifests[] |
+select(.platform.architecture=="amd64") | .digest'` and pin that digest.
 
 - Ports **389** (ldap) and **636** (ldaps) publish on the LAN
   (`PublishPort=389:3389`, `636:6636`
@@ -393,9 +392,10 @@ downstream —
 - `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
   every human account).
 - **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
-  debo, freya, mitch, eric). The pks in this page are confirmed against
-  `authentik_core_user` on 2026-10-18: ridley 13, kassie 14, alex 15,
-  artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21, mitch 22, eric 23 —
+  debo, freya, mitch, eric, jason). The pks were confirmed against
+  `authentik_core_user` on 2026-10-18 and the table re-read 2026-11-04: ridley
+  13, kassie 14, alex 15, artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21,
+  mitch 22, eric 23, **jason 24 (created 2026-11-04)** —
   and every one carries the `<user>@franksdank.cloud` address the template
   prescribes. Creation dates (2026-10-07 → 2026-10-15) were not re-checked
   - (unverified 2026-10-18)*. Template: `username` +

serverAppConfigs-docs/Infrastructure/Caddy.md

@@ -4,7 +4,7 @@ description: "Caddy is the single reverse proxy for both hosts."
 tags:
   - infrastructure
   - caddy
-timestamp: 2026-08-29
+timestamp: 2026-11-03
 ---
 
 # Caddy
@@ -90,7 +90,111 @@ pihole.franksdank.cloud {
 
 After rendering, the line reads `dankuser <hash>` (one complete hash) — `tests/deployment_flow.bats:96-99` pins the `dankuser` name and the single-hash form, and `tests/caddyfile.bats:75` pins the bare `dankuser REPLACE_WITH_HASH` template. Do not put a `$2a$14$` prefix in the template: it would render a double-prefixed, invalid hash (2026-08-22 incident — every basicauth vhost returned 401 for every password; `tests/caddyfile.bats:67-72` forbids the prefix). `tests/deployment_flow.bats` pins the render contract.
 
-Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile:14-24,283-423`). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), and the media/*arr vhosts. Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik (`dankclaw/Caddyfile:122-132`). See [[Homepage]] and [[Authentik]].
+Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`, `notebook.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile`: the header comment and the two internal listeners). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), the media/*arr vhosts, and `audiobooks.` — Audiobookshelf enforces its own auth (local accounts plus authentik OIDC) and must answer native app clients, podcast RSS and OPDS, which the chain structurally cannot deliver: it answers a client that solves no JavaScript proof-of-work with challenge HTML wherever that client expects JSON or a socket payload ([[Audiobookshelf]]). Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik. See [[Homepage]] and [[Authentik]].
+
+## Guard a new vhost
+
+Order is the whole procedure: the IdP row has to exist **before** the route flips, or
+authenticated traffic reaches the outpost with no matching provider and the user gets
+authentik's error page instead of the app.
+
+1. **Provider first.** A proxy provider plus an application, claimed by the embedded
+   outpost. No blueprint owns this state — it lives only in the IdP's database, which is
+   why it is easy to forget:
+
+   ```
+   ssh dankclaw 'podman exec -i authentik-server ak shell' <<'PY'
+   from authentik.providers.proxy.models import ProxyProvider
+   from authentik.core.models import Application
+   from authentik.outposts.models import Outpost
+   NAME, EXT = "example", "https://example.franksdank.cloud/"
+   pp = ProxyProvider.objects.filter(name=f"{NAME}-proxy").first()
+   if pp is None:
+       pp = ProxyProvider.objects.create(name=f"{NAME}-proxy", external_host=EXT,
+                                         internal_host="http://caddy:8017")
+   if not Application.objects.filter(provider=pp).exists():
+       Application.objects.create(name=f"{NAME.title()} Guard", slug=f"{NAME}-guard",
+                                  provider=pp)
+   o = Outpost.objects.get(managed="goauthentik.io/outposts/embedded")
+   o.providers.add(pp)   # an unclaimed provider is invisible to the outpost
+   print(pp.name, pp.external_host, o.providers.count())
+   PY
+   ```
+
+   `external_host` is the exact vhost URL **with the trailing slash**: the embedded
+   outpost routes on the ORIGINAL `Host` header alone, so anything else matches zero
+   providers. Never print a provider's fields wholesale — a proxy provider carries a
+   client secret, and it lands in whatever transcript the command runs in.
+
+   `ProxyProvider.objects.create()` is not the API: the serializer's
+   `set_oauth_defaults()` never runs, so the row lands with `redirect_uris` and
+   `grant_types` empty, and the guard fails *before* any credential — authentik's
+   "Redirect URI Error" page, then a bodiless HTTP 400 once the URIs are filled
+   by hand. Run `scripts/authentik-proxy-provider-defaults.sh` after the snippet
+   (idempotent, covers every provider; `--check` is the census); law and failure
+   signatures in [[Authentik Outpost Reference]].
+2. **Then the route.** The public site block proxies the whole request to `anubis:8923`
+   with `header_up X-Real-Ip {remote_host}`, and the post-auth `http://:8017` listener
+   gains a `@<name> host <vhost>` handler pointing at the app — copy `romm`/`files`,
+   which are the same shape including a Dankserver LAN-port upstream.
+   An app that derives its own origin from the request needs the scheme pinned:
+   Audiobookshelf checks every OIDC callback against `x-forwarded-proto://Host`; this
+   listener is fed over plain http so Caddy's own `X-Forwarded-Proto` is `http`, and the
+   app answers a 400 "Invalid callback URL - must be same-origin" for its own https
+   callback (measured 2026-11-03, while this vhost was still guarded — it is exempt
+   now, and the same pin sits on its edge block instead).
+   The edge for a guarded name is TLS-only, so `header_up X-Forwarded-Proto
+   https` is the truthful value ([[Audiobookshelf]]).
+3. **Then the name and the audit.** A `dankclaw/pihole/addn-hosts` line (the `pihole`
+   seam fails the build if a Caddyfile vhost has no record), a Porkbun A record if the
+   name must resolve off-LAN ([[LAN-DNS]]), and the subdomain added to `GUARDED` in
+   `tests/e2e/specs/14-guard.spec.ts` with an app-specific leak marker — that spec is
+   the deploy gate's proof that the guard, not the app, answers an anonymous client.
+
+## Unguard a vhost
+
+The same three artifacts, in the opposite order. Getting it wrong takes the app
+down for browsers while looking healthy:
+
+1. **Route first.** Move the public site block off `anubis:8923` to a direct
+   `reverse_proxy` (keeping any origin-pinning `header_up` the app needs), delete
+   the `@<name>` handler from the `http://:8017` listener, and deploy. Traffic
+   stops entering the auth listener at `:8016`, so the IdP row goes unused — and
+   nothing breaks.
+2. **Then the IdP rows**, once the route is verified:
+
+   ```
+   ssh dankclaw 'podman exec -i authentik-server ak shell' <<'PY'
+   from authentik.providers.proxy.models import ProxyProvider
+   from authentik.outposts.models import Outpost
+   pp = ProxyProvider.objects.filter(name="audiobooks-proxy").first()
+   if pp:
+       o = Outpost.objects.get(managed="goauthentik.io/outposts/embedded")
+       o.providers.remove(pp)
+       app = pp.application          # the "<Name> Guard" application row
+       if app:
+           app.delete()
+       pp.delete()
+   print("providers now claimed:", Outpost.objects.get(
+       managed="goauthentik.io/outposts/embedded").providers.count())
+   PY
+   ```
+
+   Deleting the provider **before** the route flips is the wrong order: the
+   outpost then matches zero providers for a Host that still arrives, and every
+   browser gets authentik's error page instead of the app.
+3. **Then the audit lists.** Drop the subdomain from `GUARDED` in
+   `tests/e2e/specs/14-guard.spec.ts` (and its leak marker), and replace it
+   there with an exemption assertion — the deploy gate must keep proving the
+   exemption is deliberate, the way `plex` does: app bytes to an anonymous
+   client, zero challenge HTML, and the app's own `401` on an unauthenticated
+   API call rather than the gate's `200`.
+
+What the exemption costs, said plainly: the app's login endpoint becomes
+internet-visible with no rate limiter in front of it unless the edge carries one
+(this Caddy build has no `rate_limit` module) or the app ships a lockout (ABS
+2.37.2 does not). That is the same exposure `jelly.`, `plex.` and the *arr
+vhosts already live with; the mitigation is the credentials.
 
 ## Operations
 
@@ -113,7 +217,10 @@ Certificates and ACME state live in the caddy data volume (`data/caddy/`). Back
 - A bare compose container is **not** boot-persistent: after the 2026-08-26 reboot Caddy stayed down for hours because `restart: unless-stopped` does not survive shutdown's explicit stop. It now runs only from its quadlet unit, and the repo has no `dankclaw/compose.yaml` left to re-add it to (`ls`, 2026-10-18).
 - `admin off` disables the config file watcher: rendered changes then sit on disk while Caddy serves its stale in-memory config until the container is recreated, and nothing logs an error (2026-08-22: a broken basicauth hash survived several deploys this way). `tests/caddyfile.bats` forbids `admin off`.
 - The basicauth template must stay a bare `REPLACE_WITH_HASH` token (no `$2a$14$` prefix) — the rendered value is the complete hash. A double prefix silently rejects every password with 401; no Caddy error is logged.
-- A public vhost needs three things: a Pi-hole `addn-hosts` line, a Porkbun A record, and the Caddyfile block. See [[LAN-DNS]]. The deliberate exception is an **internal-only** name: `hermes.franksdank.cloud` and `hermesadmin.franksdank.cloud` are in `addn-hosts` and the Caddyfile but have no public record, which is what keeps them LAN-only — DNS-01 issuance needs no public A record (`dankclaw/Caddyfile:173-179`).
+- **Exposure has exactly two gates: the router's 80/443 forward to Caddy, and Caddy's vhost list.** Caddy matches on SNI/Host, so *every* name in the Caddyfile already answers a WAN client that sends that name — a public A record buys **discoverability, not access** (measured 2026-11-04). A published name needs a Pi-hole `addn-hosts` line, a Porkbun A record and the Caddyfile block ([[LAN-DNS]]); DNS-01 issuance needs no public record, so an **internal-only** name is one with no A record. Posture measured 2026-11-04: the whole authentik-guarded set is published — `www.`, `search.`, `romm.`, `files.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `notebook.` — while the names with no public record are the self-authenticating exempt ones (`pihole.`, `hermes.`, `hermesadmin.`, `prowlarr.`, `radarr.`, `lidarr.`, `readarr.`, `qbittorrent.`) and the legacy redirects (`dankclaw.`, `dankserver.`). Every A record is a copy of the WAN address, which is dynamic — republish after an IP change.
+- **A missing DNS record is not a gate: the names are public anyway.** Caddy issues one certificate per name and every issuance is published in Certificate Transparency logs — `crt.sh` for `%.franksdank.cloud` lists the live *and* retired set (`sonarr.`, `comfyui.`, `chat.`, `jellyfin.`, the whole *arr stack), so "LAN-only because there is no A record" is fiction and a stranger can send the Host. A household-only service needs a **source gate in its vhost**, which is what the *arr stack and qBittorrent now carry: a LAN client proxies through, anything off-LAN gets `404`. Keep those five gated even though they have no A record.
+- **The LAN gate is `remote_ip`, and it reads the true client.** The G36 DNATs the 80/443 forward straight to Caddy with no proxy hop, so `@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12` sees either a LAN address or the internet address — never the router's. Verify the split without waiting for a WAN client: a LAN request must reach the upstream (a dead upstream still proves it passed the gate, because the answer is its `502`, not the gate's `404`), and an external fetcher must get the `404`.
+- **A new record looks like it failed for up to the SOA's negative TTL.** The zone's negative cache is 1800 s, so a resolver that asked for the name *before* the create keeps answering NXDOMAIN (1.1.1.1 did, while Quad9 and the authoritative NS had the record seconds after the create) — check `dig @<authoritative NS>` before re-creating anything.
 - The G36 has no NAT loopback, so LAN clients must resolve the name to the LAN IP. That is the Pi-hole role in [[LAN-DNS]]. *(unverified 2026-10-18: hairpin behaviour is only observable from a WAN-side client; nothing on the LAN or in this repo can prove it)*
 - Caddy serves HTTPS on 443 and HTTP on 80 (redirects). The router needs no 80/443 port forward for LAN use.
 - `transport http {versions h2c 1.1}` on a vhost makes Caddy send an h2c

serverAppConfigs-docs/Infrastructure/Homepage.md

@@ -28,11 +28,12 @@ descriptions and a `docs` link to `/docs#models`.
 
 **Sections + WIP split (2026-09-21):** both pages now group their panels
 under section headings rendered by shared `renderSection()`/`renderPanel()`
-components in `app.js`. The user page orders **AI, Web Apps, Media Library
-Apps**; the admin page orders **Media Library Apps, Observability, AI,
-Infrastructure**. The WIP badge split into **WIP: OPEN** (work in progress)
-and **WIP: CLOSED** (work finished by decision — e.g. a card that is
-deliberately unprobeable) via the `wip` value (`"open"` / `"closed"`).
+components in `app.js`. The user page orders **Web Apps, Media Library
+Apps, AI, Work in progress** (the three not-open-yet apps moved under their
+own heading, below AI, on 2026-11-04); the admin page orders **Media Library
+Apps, Observability, AI, Infrastructure**. The WIP badge split into **WIP:
+OPEN** (work in progress) and **WIP: CLOSED** (work finished by decision — e.g.
+a card that is deliberately unprobeable) via the `wip` value.
 Tautulli (Plex statistics, `192.168.0.235:8181`) joined the admin page's
 Media Library Apps section.
 
@@ -314,9 +315,10 @@ flowchart TD
 ## Service classification
 
 **User services** (root page), in section order:
-- **AI** — OpenWebUI, and the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot — status + docs link, no Open link).
 - **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
-- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
+- **Media Library Apps** — Plex, Jellyfin, Audiobookshelf (moved up from the admin page 2026-11-03, once Authentik SSO made it a household app).
+- **AI** — OpenWebUI, the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot) and `frank-tts`, the cloned-voice endpoint (moved up from the admin page 2026-11-04 so the household sees whether the voice door is up). All four are status + docs link, no Open link; the fused **Model Information** panel renders under this heading.
+- **Work in progress** — Calibre-Web, RomM, Seerr. WIP badge, no Open link, no household account yet: the ask-Frank door. See the docs page's *Three doors that are not open to the household yet*.
 
 **Admin services** (`/admin/`), in section order:
 - **Media Library Apps** — the *arr stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr), qBittorrent, and Tautulli — live on FrankBook-Pro and Dankserver, operator-only: no vhosts (Tautulli included), direct IP:port access only. See [[FrankBook-Pro]].
@@ -324,6 +326,12 @@ flowchart TD
 - **AI** — Hermes, ComfyUI.
 - **Infrastructure** — Caddy, dnsmasq, Gluetun, and the low-level backends (mariadb, valkey, Postgres). The backends show status only. They get a link only where a UI is meaningful.
 
+**The registry is the authority.** `shared/homepage/services.json` decides which
+page a card renders on (`category`), whether it shows a WIP badge instead of live
+status (`wip`), and whether it gets an Open link (`showLink` + `url`). The two
+lists above are a locator only; when they disagree with the registry, the page is
+the bug — fix the page.
+
 ## Layout
 
 ```
@@ -426,15 +434,22 @@ agent knowledge base (`~/.pi/agent/docs/caddy/caddyfile.md`).
 
 - `category` — `user` or `admin`. Selects the page.
 - `section` — the heading the panel renders under. Must be one of the page's
-  `SECTION_ORDER` entries (user: AI, Web Apps, Media Library Apps; admin:
-  Media Library Apps, Observability, AI, Infrastructure), or the panel is
-  silently dropped from the page.
+  `SECTION_ORDER` entries (user: Web Apps, Media Library Apps, AI, Work in
+  progress; admin: Media Library Apps, Observability, AI, Infrastructure), or
+  the panel is silently dropped from the page.
 - `url` — the "Open" link (the Caddy site).
 - `docs` — the per-service docs page (`/docs/<svc>/`).
 - `probe` — the backend `host:port` the checker probes.
 - `method` — `tcp` or `http`. The probe type for this service (Q10).
-- `wip` — `false`, or `"open"` / `"closed"`. When set, the badge shows
-  **WIP: OPEN** or **WIP: CLOSED** and the checker skips the probe.
+- `wip` — `false`, or `"open"` / `"closed"` / `"down"`. When set, the badge shows
+  **WIP: OPEN** / **WIP: CLOSED** / **WIP: DOWN** and the checker skips the probe.
+- `model` — present only on Model Information cards. Such a card renders in the
+  fused `renderModelInfo()` panel (`mi-*` styles) instead of the app grid, and its
+  `description` is unused. `unit` is the StarCraft identity: art
+  `/img/unit-<unit>.jpg`, accent `.mi-<unit>`, voice lines from `/sfx/`. A card
+  with no `unit` is a tool (today: `frank-tts`) — art falls back to the card's own
+  `art`, the accent is `.mi-tool`, and a `tagline` with no `sfx` renders as plain
+  text rather than a play button.
 
 **A card must be probeable from the monitor's host, or it must be `wip`.** The
 checker runs on Dankclaw, so its reach is: that host's own loopback, its own

serverAppConfigs-docs/Infrastructure/Pi-hole.md

@@ -77,7 +77,7 @@ Both prerequisites survive the dnsmasq → Pi-hole swap. Do not remove them.
 | Port 53 `bind: permission denied` crash loop after a reboot | sysctl conflict: the value must be 53 at boot, and remember `/etc/sysctl.conf` is applied **after** all of `sysctl.d/`. Redeploy pihole — `deploy_pihole` reinstalls the drop-in, neutralizes conflicting drop-ins, and comments out a stale `/etc/sysctl.conf` line |
 | Single-label name (for example `dankclaw`) answers `127.0.1.1` | The rootless container `/etc/hosts` carries the host entry (`127.0.1.1 <host>`), and FTL serves it. Use the FQDN, or add an `/etc/hosts` entry on the client |
 | Every `franksdank.cloud` name is NXDOMAIN for LAN clients while external names still answer | FTL holds no local records: `getent hosts <name>` on Dankclaw comes back empty → `podman exec pihole pihole restartdns reload` (the 2026-10-19 pitfall under **Records**) |
-| One client gets nothing while every other client is fine | FTL rate-limiting that client: `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`. A `10.0.2.x` client address is a pasta container's DNS forwarder, not a machine |
+| One client gets nothing while every other client is fine | FTL rate-limiting that client: `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`. A `10.0.2.x` client address is a pasta container's DNS forwarder, not a machine — and **every** rootless container on that host shares the one address, so a single chatty container throttles DNS for all of them (measured 2026-10-31: 8 throttle events in a day, all `10.0.2.100`) |
 
 ## See Also
 

serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

@@ -197,23 +197,18 @@ and PerfOpt is now a bonus nobody is waiting for rather than the reason to flip.
 **Post-boot gates.** 0 failed units, `systemd` running. 32 IOMMU groups — 31
 `identity` + 1 `DMA`, and that one belongs to the Genesys Logic SD card reader
 (`c2:00.0`, a kernel quirk, not a fleet device). The iGPU (`c5:00.0`) sits in group
-20 `identity` with ATS capability; interrupt remapping is enabled. frank-dragoon's
-boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + 5.5
-GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
-depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
-answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
-slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
-boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
-is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working)
-see [[frank-dragoon]].)
+20 `identity` with ATS capability; interrupt remapping is enabled. The flip moved
+frank-dragoon's boot ledger by nothing measurable — its shape and every number in it are
+owned by [[frank-dragoon]] — and Gitea, the Caddy edge and Grafana all answered 200.
 
 **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
 Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
 amdxdna_accel_driver 0.7.0 … on minor 0`, IOMMU group 26 `identity`, PASID cap width
 10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank`
 is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
-`GroupAdd=keep-groups`. **No NPU endpoint is deployed** —  stays the
-history of the retired endpoint; what changed is only its precondition.
+`GroupAdd=keep-groups`. The flip brought the device back, not a service: no NPU chat
+endpoint was deployed then. The XDNA2 lane returned to work a day later as frank-dragoon's
+small-model lane ([[frank-dragoon]]) — the shape and its laws are owned there.
 
 **The `pp_dpm_fclk` pin (level 7, 2000 MHz) went in on 2026-10-19**, with the first thing that needed it: the
 vendor's root host unit, installed and enabled by `frank-dragoon`'s deployer while its NPU lane is named. Pinning

serverAppConfigs-docs/LLM/frank-archon.md

@@ -20,7 +20,7 @@ timestamp: 2026-10-19
 The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
 path) driving `podman run` straight from the digest-pinned image under a **root** systemd
 unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
-PRO R9700s. Capacity contract **2 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
+PRO R9700s. Capacity contract **4 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
 is the **demand signal** the budget resolver sizes the cache against, not a KV allocation (Laws).
 A payload swap is a `MODEL_FILE` line plus a re-download — see Eras.
 
@@ -525,6 +525,21 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 - `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
   the disk tier silently never opens. The tier does survive a payload swap when arch and
   tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).
+- **A radiance crash reads as a clean exit unless you know where to look, and the engine never
+  says why.** The unit's `ExecStart` ends in `exec podman wait`, so `systemctl show -p
+  ExecMainStatus` reports **0** after a container death — that is `podman wait`'s own status, not
+  the engine's — and `Result=success` beside `NRestarts=1` is the *normal* fingerprint of a
+  crash. The container's real code is the bare digit `podman wait` prints to stdout
+  (`serve-frank-archon.sh[NNNN]: 1`). The engine's fatal path writes **nothing** to its log: no
+  stderr line, no message before the exit and no core. The only copy of the reason is the error
+  the client received — `the engine stopped on an error and is not serving (<errno>)`, where the
+  parenthesis is the failing call's errno text (`invalid argument` = EINVAL, 2026-11-04). So
+  grep the *client* logs before concluding the journal is empty: omp keeps the provider error in
+  `~/.omp/logs/omp.<date>.<pid>.log` (`agent turn ended with provider error`), and OpenWebUI
+  shows it as the chat's error toast. Seen once (2026-11-04, restart counter 1 after 3d20h up):
+  three concurrent triple-digit-thousand-token agent turns prefilled at once, prefill rate
+  collapsed, all in-flight streams failed at the same instant, then a silent teardown — kernel
+  logged only the KFD SVM workqueue churning, with no OOM, GPU hang, MCE or swap pressure.
 
 ### Dead ends — do not re-try