Update AGENTS.md with target-specific branching and service status

Add AGENTS.md, deploy script, update README with service status and file locations
Update README with validation instructions and performance data
2026-08-09 10:37:57 -07:00 · 2026-08-09 09:47:24 -07:00 · 2026-08-08 21:46:00 -07:00 · 2026-08-08 21:42:37 -07:00 · 2026-08-08 21:29:23 -07:00 · 2026-08-08 17:28:00 -07:00
542 changed files with 2467 additions and 60979 deletions
@@ -1,71 +0,0 @@
 # Tunables for frank-default-small (Tiel-Coder-35B-A3B UD-Q6_K_XL,
 # qwen35moe hybrid MoE, Dankserver, RTX 3090).
 #
 # Runtime: upstream llama.cpp (latest master, CUDA build with
 # -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86) on RTX 3090
 # 24 GB; Ryzen 7 7700X; 62 GB RAM.  Replaced the ik_llama.cpp Ornith
 # CRACK Q8_0 deployment on 2026-09-01.
 #
 # Model: peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF, UD-Q6_K_XL quant
 # (30 GB).  Architecture: qwen35moe — 40 layers, full_attention_interval=4
 # (10 attention + 30 SSM/Mamba), 256 experts / 8 used, 2 KV heads × 256
 # dim, embedding 2048, rope dimension_count 64, rope freq_base 10 000 000.
 # Text-only (no mmproj).  Built-in chat template in the GGUF.
 #
 # Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
 # Non-expert weights (~1.6 GB) fit entirely on the 24 GB card.
 #
 # KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per slot
 # × 3 parallel) and 10 attention layers, the KV cache is ~2 GB — the
 # requested 2 GB VRAM buffer.
 #
 # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
 # ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
 # incident). This box is NVIDIA/CUDA, but the rule keeps the env portable.
 # Model file (UD-Q6_K_XL, 30 GB, text-only MoE hybrid).
 MODEL_DIR=/home/dank/models/tiel-coder-35b
 MODEL_FILE=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
 # Upstream llama.cpp llama-server (CUDA build, latest master).
 # Built with -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86
 # using the staged CUDA 13.2 toolkit at /home/dank/cuda13-stage.
 LLAMA_SERVER=/home/dank/llama.cpp/build/bin/llama-server
 ALIAS=frank-default-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7770
 # Total context budget across all parallel slots.  200700 / 3 = 66 900
 # per slot.  The model's context_length is 262 144, so 66 900 is well
 # within range.
 CTX_SIZE=200700
 PARALLEL=3
 # Expert offload: all MoE expert weights pinned to CPU RAM.
 CPU_MOE=1
 # All non-expert layers on GPU (auto-fit will determine the exact count;
 # with --cpu-moe the non-expert footprint is ~1.6 GB, leaving ample room).
 N_GPU_LAYERS=999
 # KV cache: Q8_0 for both K and V.  ~2 GB total for 200700 ctx × 3 slots
 # × 10 attention layers × (2 KV heads × 256 dim × 1 byte) × 2 (K+V).
 CACHE_TYPE_K=q8_0
 CACHE_TYPE_V=q8_0
 VERBOSITY=2
 # Prefill batch sizing.
 BATCH_SIZE=4096
 UBATCH_SIZE=1024
 # Sampling.
 TEMPERATURE=1.0
 TOP_K=20
 TOP_P=0.95
 MIN_P=0.0
 # Compute threads lowered so other processes keep a core.
 THREADS=7
 THREADS_HTTP=8
@@ -1,16 +0,0 @@
 [Unit]
 Description=llama.cpp Tiel-Coder-35B-A3B Q6_K_XL qwen35moe hybrid MoE (dankserver, frank-default-small)
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=simple
 User=dank
 Group=dank
 Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-small.env"
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=multi-user.target
@@ -1,126 +0,0 @@
 # Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
 # STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
 # Strix Halo gfx1151, unified memory).
 #
 # Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
 # qwen4exp-on-rocmfpx-d3ca537 patch applied — the model card's pinned
 # runtime: ROCmFP4 tensor types AND the qwen4exp architecture in one
 # tree; stock llama.cpp and charlie's fork alone cannot load this GGUF).
 # Built per the card recipe:
 #   cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
 # at /home/dank/GitPackages/kingjones-ROCmFPX/build.
 # Replaced the agentionai ROCmFP4-FAST imatrix deployment (charlie's
 # build-qwen38 runtime) on 2026-08-30.
 #
 # Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
 # 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
 # at layer 1), Q4_0_ROCMFP4_STRIX (file_type 105), 113.47 GiB single
 # file. STRIX is the top quality tier: PLE table at Q8_0 (50.66 GiB),
 # token embeddings + lm head at Q6_K. The PLE table NEVER enters GPU
 # memory — it streams from the file through the OS page cache. Two hard
 # card constraints:
 #  1. mmap MUST stay enabled (default) — --no-mmap forces the 50.66 GiB
 #     table into anonymous memory and the cgroup killer takes the
 #     process with nothing in the server log (only dmesg).
 #  2. Do NOT pin the PLE tensor to CPU with -ot — measured gen fell
 #     ~23 -> 13.4 tok/s; the kernel streams it better.
 # NOTE: the card's own build_attn_qsa still calls ggml_fill on the
 # F16-cast kq mask (the path that aborted our charlie build with
 # --flash-attn on before scripts/patches/qwen4exp-qsa-f32-mask.patch).
 # If a clean kingjones-tree build aborts graph build with
 # GGML_ASSERT(a-type == GGML_TYPE_F32), port that fix (F32 base mask +
 # cast at the attention call) into this tree and rebuild.
 #
 # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
 # incident). llama.cpp reads --host/--port/--temp as CLI flags, never as
 # env vars.
 # kingjones30/ROCmFPX runtime binary (card recipe build).
 LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
 # Model package from
 # https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
 # Layout: single unsharded GGUF, text-only (the repo ships no mmproj;
 # the vision tower below is borrowed from the agentionai FAST package —
 # same base model, f16 — kept only if it loads clean, drop
 # MMPROJ_PATH if it does not).
 MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
 MODEL_FILE=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
 MMPROJ_PATH=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf
 # Qwen-Sharp chat template (tracked in the repo, staged on the host). The
 # Sharp template drives thinking on by default (medium effort); the server
 # emits reasoning in DeepSeek format so clients receive reasoning_content
 # separately instead of literal thinking tags in content.
 CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
 CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
 # Alias + network.
 ALIAS=frank-dragoon
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7770
 # Serving: 401408-token context, TWO slots (parallel 2), batch 4096 /
 # ubatch 512, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
 # prompt cache. --fit off: auto-fit segfaults on this model's
 # hyper-connection/indexer config; -ngl 999 puts every layer on the
 # SVM-backed GPU pool. ubatch 512 matches the card's re-measurement
 # (-ub 1024/2048/4096 showed no effect; -t 32 vs 16 is noise).
 #
 # CTX_SIZE=401408 / PARALLEL=2: carried over from the FAST deployment
 # where it loaded clean (~74 s, health 200). The card measures the full
 # 262144 window at ~71.7 GiB GTT at load on the same 128 GB board; QSA
 # caps KV growth so the extra window is cheap. KV footprint scales
 # with the 12 attention layers only. Verify load on first STRIX boot.
 CTX_SIZE=401408
 PARALLEL=2
 BATCH_SIZE=4096
 UBATCH_SIZE=512
 NUM_THREADS=16
 THREAD_BATCH=16
 CACHE_TYPE_K=f16
 CACHE_TYPE_V=f16
 CTX_CHECKPOINTS=16
 CACHE_RAM=8192
 SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
 SPLIT_MODE=none
 FIT=off
 NGL=999
 FLASH_ATTN=on
 # Sampling (card instruct table, non-thinking).
 TEMPERATURE=0.7
 TOP_K=20
 TOP_P=0.8
 MIN_P=0.0
 PRESENCE_PENALTY=1.5
 # Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
 # keep the reasoning trail in the conversation). DeepSeek format projects the
 # reasoning into the API's reasoning_content field instead of literal thinking
 # tags.
 REASONING=on
 REASONING_FORMAT=deepseek
 # Strix Halo unified memory + native gfx1151.
 HSA_OVERRIDE_GFX_VERSION=11.5.1
 VERBOSITY=4
 # Speculative decoding: disabled. The card's Usage block ships no spec
 # flags. The Flash-Next MTP heads do not load on this loader (requires
 # per-layer blk.N.hc_attn_norm; every head carries only blk.48 —
 # verified 3x on build-qwen38). DRAFT_MODEL_* point at the retired
 # agentionai FAST head and are unused while SPEC_TYPE=none.
 DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
 DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
 SPEC_TYPE=none
 SPEC_DRAFT_N_MAX=3
 SPEC_DRAFT_N_MIN=0
 SPEC_DRAFT_P_MIN=0.75
 SPEC_DRAFT_NGL=999
 SPEC_DRAFT_DEVICE=ROCm0
 SPEC_DRAFT_TYPE_K=f16
 SPEC_DRAFT_TYPE_V=f16
@@ -1,16 +0,0 @@
 [Unit]
 Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=simple
 User=dank
 Group=dank
 Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=multi-user.target
@@ -1,23 +0,0 @@
 # Tunables for frank-npu (FastFlowLM NPU endpoint, Dankclaw)
 # Runtime: FLM 1.0.1 (/usr/bin/flm) on the XDNA2 NPU (50 TOPS). Boot tag is
 # nuextract3:4b (NuExtract3-4B-NPU2, Qwen3.5-4B skeleton, structured-extraction
 # model); serve routes every request by the exact FLM registry tag in the
 # request's model field, so any registered tag (gemma4-it:e4b, qwythos:9b, ...)
 # is reachable on this port regardless of the boot tag. The NPU is a
 # single-lane device: generations serialize FIFO, and the first request after a
 # model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
 # pool shared with frank-dragoon (iGPU).
 FLM_MODEL=nuextract3:4b
 FLM_HOST=0.0.0.0
 FLM_PORT=7771
 # turbo = max NPU power mode (xrt-smi configure --pmode turbo at serve start).
 # No other tenant uses the NPU on this host, so the top mode costs nothing to
 # neighbours; drop to `performance` if iGPU power-budget contention ever shows.
 FLM_PMODE=turbo
 # FLM_ASR=1 additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes
 # /v1/audio/transcriptions. NuExtract3 answers text-only; audio stays off.
 FLM_ASR=0
 # -1 = no --ctx-len flag: every pack loads with its own trained context
 # (nuextract3:4b -> 32768, gemma4-it:e4b -> 131072). A global override would
 # over-allocate KV for the small packs.
 FLM_CTX_LEN=-1
@@ -1,17 +0,0 @@
 [Unit]
 Description=FastFlowLM NPU sight+sound inference (frank-npu) on Dankclaw
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=simple
 User=dank
 Group=dank
 Environment="FRANK_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-npu.env"
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-npu.sh
 LimitMEMLOCK=infinity
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=multi-user.target
@@ -1,180 +0,0 @@
 #!/usr/bin/env bash
 # Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
 # frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
 # Dankclaw (Strix Halo gfx1151, unified memory).
 #
 # Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
 # model card's qwen4exp-on-rocmfpx patch) built at
 # /home/dank/GitPackages/kingjones-ROCmFPX/build (HIP, gfx1151,
 # ROCm 7.2.4, card recipe). The ROCmFP4 quant types plus the qwen4exp
 # architecture must be in one tree; stock llama.cpp cannot load this
 # GGUF.
 #
 # Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
 # 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
 # at layer 1) quantized Q4_0_ROCMFP4_STRIX, 113.47 GiB single file. The
 # 50.66 GiB PLE n-gram table stays OUT of GPU memory and streams via
 # mmap from the OS page cache — never add --no-mmap or -ot for it.
 #
 # Profile: card sampling table (temp 0.7, top_p 0.80, top_k 20, min_p 0.0,
 # presence 1.5 — non-thinking instruct) mixed with the validated serving
 # profile: ctx 401408 x 2 slots, b 4096 / ub 512, t 16 / tb 16, f16 KV,
 # fa on (if the graph build aborts on the QSA F16 mask, see the note in
 # the env file: port scripts/patches/qwen4exp-qsa-f32-mask.patch into
 # this tree and rebuild), cont-batching, prompt cache + 8 GiB cache-ram
 # + idle-slot cache, 16 ctx checkpoints, --fit off (auto-fit segfaults
 # on qwen4exp hyper-connection config). Speculative decoding off: the
 # Flash-Next MTP heads do not load on this loader; the card ships no
 # spec flags.
 set -euo pipefail
 SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
    # shellcheck disable=SC1090
    source "$ENV_FILE"
    set +a
 fi
 # --- defaults (used when the env file does not set them) -------------------
 : "${LLAMA_BIN:=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
 : "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
 : "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
 : "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
 : "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
 : "${CHAT_TEMPLATE_KWARGS:={}}"
 : "${ALIAS:=frank-dragoon}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7770}"
 : "${CTX_SIZE:=401408}"
 : "${PARALLEL:=2}"
 : "${BATCH_SIZE:=4096}"
 : "${UBATCH_SIZE:=512}"
 : "${NUM_THREADS:=16}"
 : "${THREAD_BATCH:=16}"
 : "${CACHE_TYPE_K:=f16}"
 : "${CACHE_TYPE_V:=f16}"
 : "${CTX_CHECKPOINTS:=16}"
 : "${CACHE_RAM:=8192}"
 : "${SLOT_SAVE_PATH:=$SCRIPT_DIR/slot-state}"
 : "${SPLIT_MODE:=none}"
 : "${FIT:=off}"
 : "${NGL:=999}"
 : "${FLASH_ATTN:=on}"
 : "${TEMPERATURE:=0.7}"
 : "${TOP_K:=20}"
 : "${TOP_P:=0.8}"
 : "${MIN_P:=0.0}"
 : "${PRESENCE_PENALTY:=1.5}"
 : "${REASONING:=on}"
 : "${REASONING_FORMAT:=deepseek}"
 : "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
 : "${SPEC_TYPE:=none}"
 : "${DRAFT_MODEL_DIR:=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST}"
 : "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
 : "${SPEC_DRAFT_N_MAX:=3}"
 : "${SPEC_DRAFT_N_MIN:=0}"
 : "${SPEC_DRAFT_P_MIN:=0.75}"
 : "${SPEC_DRAFT_NGL:=999}"
 : "${SPEC_DRAFT_DEVICE:=ROCm0}"
 : "${SPEC_DRAFT_TYPE_K:=f16}"
 : "${SPEC_DRAFT_TYPE_V:=f16}"
 MODEL="$MODEL_DIR/$MODEL_FILE"
 for f in "$LLAMA_BIN" "$MODEL" "$CHAT_TEMPLATE_FILE"; do
    if [[ ! -r "$f" ]]; then
        echo "missing or unreadable required artifact: $f" >&2
        exit 1
    fi
 done
 if [[ ! -x "$LLAMA_BIN" ]]; then
    echo "missing executable runtime: $LLAMA_BIN" >&2
    exit 1
 fi
 # --- speculative decoding (MTP draft head) ---------------------------------
 # The MTP draft head is a separate GGUF sidecar loaded via --spec-draft-model
 # (alias --model-draft).  ngram-only specs do not need a draft file.
 # Set SPEC_TYPE=none in the env to disable speculative decoding entirely.
 SPEC_ARGS=()
 if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
    SPEC_ARGS=(
        --spec-type "$SPEC_TYPE"
        --spec-draft-n-max "$SPEC_DRAFT_N_MAX"
        --spec-draft-n-min "$SPEC_DRAFT_N_MIN"
        --spec-draft-p-min "$SPEC_DRAFT_P_MIN"
    )
    if [[ "$SPEC_TYPE" == *draft-mtp* ]]; then
        DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
        if [[ ! -r "$DRAFT_MODEL" ]]; then
            echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
            exit 1
        fi
        SPEC_ARGS+=(
            --spec-draft-model "$DRAFT_MODEL"
            --spec-draft-ngl "$SPEC_DRAFT_NGL"
            --spec-draft-device "$SPEC_DRAFT_DEVICE"
            --spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
            --spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
        )
    fi
    if [[ -n "${SPEC_DRAFT_P_SPLIT:-}" ]]; then
        SPEC_ARGS+=(--spec-draft-p-split "$SPEC_DRAFT_P_SPLIT")
    fi
 fi
 # The STRIX repo ships no mmproj; this is the agentionai FAST package's
 # f16 vision tower for the same base model. Passed only when readable.
 MMPROJ_ARGS=()
 if [[ -n "${MMPROJ_PATH:-}" && -r "${MMPROJ_PATH:-}" ]]; then
    MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
 fi
 SERVER_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 mkdir -p "$SLOT_SAVE_PATH"
 ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
 LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
 exec /usr/bin/env \
    ROCM_PATH="$ROCM_PATH" \
    HIP_VISIBLE_DEVICES=0 \
    LD_LIBRARY_PATH="$LIBRARY_PATH" \
    HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
    "$LLAMA_BIN" \
    --model "$MODEL" \
    --alias "$ALIAS" \
    --chat-template-file "$CHAT_TEMPLATE_FILE" \
    --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS" \
    --host "$BIND_ADDR" \
    --port "$LISTEN_PORT" \
    --jinja \
    --device ROCm0 \
    --gpu-layers "$NGL" \
    --split-mode "$SPLIT_MODE" \
    --parallel "$PARALLEL" \
    --ctx-size "$CTX_SIZE" \
    --batch-size "$BATCH_SIZE" \
    --ubatch-size "$UBATCH_SIZE" \
    --fit "$FIT" \
    --flash-attn "$FLASH_ATTN" \
    --cache-type-k "$CACHE_TYPE_K" \
    --cache-type-v "$CACHE_TYPE_V" \
    --threads "$NUM_THREADS" \
    --threads-batch "$THREAD_BATCH" \
    --ctx-checkpoints "$CTX_CHECKPOINTS" \
    --cache-ram "$CACHE_RAM" \
    --slot-save-path "$SLOT_SAVE_PATH" \
    --cache-prompt \
    --cache-idle-slots \
    --metrics \
    --temp "$TEMPERATURE" \
    --top-p "$TOP_P" \
    --top-k "$TOP_K" \
    --min-p "$MIN_P" \
    --presence-penalty "$PRESENCE_PENALTY" \
    --reasoning "$REASONING" \
    --reasoning-format "$REASONING_FORMAT" \
    --reasoning-budget -1 \
    "${MMPROJ_ARGS[@]}" \
    "${SPEC_ARGS[@]}"
@@ -1,62 +0,0 @@
 #!/usr/bin/env bash
 # Serve the FastFlowLM (FLM) sight+sound endpoint — frank-npu.
 #
 # Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
 # single-lane device: generations serialize in FIFO order, so this is a
 # background/triage endpoint, not an interactive multi-user chat. The NPU
 # draws from the unified RAM pool shared with frank-dragoon (the iGPU
 # llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
 # for the memory budget and the model-selection tradeoff.
 #
 # NOTE: FLM requires unlimited memlock. The unit sets LimitMEMLOCK=infinity.
 # NOTE: an unknown model tag does NOT 404 — /v1/chat/completions answers HTTP 200 with a
 # flat {"error":"Failed to load <tag> model!"} body and no choices; /v1/completions returns a
 # nested OpenAI-shaped server_error (code 500) also under HTTP 200. Client model IDs must be the exact FLM registry
 # tag (e.g. gemma4-it:e4b), never a service alias. Custom tags only resolve
 # when FLM_CONFIG_PATH points at the user registry that carries them and their
 # kernels are symlinked into /home/dank/.config/flm/xclbins/<PackName>.
 set -euo pipefail
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 ENV_FILE="${FRANK_NPU_ENV_FILE:-$SCRIPT_DIR/frank-npu.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
    # shellcheck disable=SC1090
    source "$ENV_FILE"
    set +a
 fi
 # --- defaults (used when the env file does not set them) -------------------
 : "${FLM_BIN:=/usr/bin/flm}"
 : "${FLM_MODEL:=gemma4-it:e4b}"
 : "${FLM_HOST:=0.0.0.0}"
 : "${FLM_PORT:=7771}"
 : "${FLM_PMODE:=turbo}"
 : "${FLM_ASR:=0}"
 : "${FLM_CTX_LEN:=131072}"
 # The user registry is a superset of the system registry (it carries the
 # custom tags too), and its model_path resolves relative to the registry file,
 # so exports keep a tag swap in the env file from needing script edits.
 export FLM_CONFIG_PATH="${FLM_CONFIG_PATH:-/home/dank/.config/flm/model_list.json}"
 export FLM_XCLBIN_PATH="${FLM_XCLBIN_PATH:-/home/dank/.config/flm/xclbins}"
 if [[ ! -x "$FLM_BIN" ]]; then
    echo "missing flm binary: $FLM_BIN" >&2
    exit 1
 fi
 # --- optional flags ----------------------------------------------------------
 OPT_ARGS=()
 if [[ "${FLM_ASR}" != "0" ]]; then
    OPT_ARGS+=(--asr "${FLM_ASR}")
 fi
 if [[ "${FLM_CTX_LEN}" != "-1" ]]; then
    OPT_ARGS+=(--ctx-len "${FLM_CTX_LEN}")
 fi
 exec "$FLM_BIN" serve "$FLM_MODEL" \
    --host "$FLM_HOST" \
    -p "$FLM_PORT" \
    --pmode "$FLM_PMODE" \
    "${OPT_ARGS[@]}"
@@ -1,87 +0,0 @@
 #!/usr/bin/env bash
 # Serve peculiar-ragdoll/Tiel-Coder-35B-A3B (UD-Q6_K_XL) as
 # frank-default-small via stock llama.cpp on Dankserver (RTX 3090).
 #
 # Runtime: upstream llama.cpp (latest master, CUDA build with
 # -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86).
 # The model is a 35B-total / 8-active MoE hybrid (qwen35moe arch) with
 # SSM/Mamba layers (full_attention_interval=4: 10 attention + 30 SSM
 # layers out of 40).  256 experts, 8 used per token.
 #
 # Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
 # Non-expert weights (attention, SSM, shared expert, embeddings, output)
 # are ~1.6 GB and fit entirely on the 24 GB RTX 3090.
 #
 # KV cache: Q8_0 for both K and V.  With CTX_SIZE=200700 (66 900 per
 # slot × 3 parallel) and 10 attention layers (2 KV heads × 256 dim),
 # the KV cache is ~2 GB — the requested 2 GB VRAM buffer.
 #
 # Model is loaded with mmap (default).  The 30 GB Q6_K_XL file is
 # memory-mapped; only accessed pages enter RAM.
 #
 # Tunables come from frank-default-small.env unless FRANK_SMALL_ENV_FILE
 # overrides that path.
 set -euo pipefail
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 ENV_FILE="${FRANK_SMALL_ENV_FILE:-$SCRIPT_DIR/frank-default-small.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
    # shellcheck disable=SC1090
    source "$ENV_FILE"
    set +a
 fi
 # --- defaults (used when the env file does not set them) -----------------
 : "${MODEL_DIR:=/home/dank/models/tiel-coder-35b}"
 : "${MODEL_FILE:=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf}"
 : "${LLAMA_SERVER:=/home/dank/llama.cpp/build/bin/llama-server}"
 : "${ALIAS:=frank-default-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7770}"
 : "${CTX_SIZE:=200700}"
 : "${PARALLEL:=3}"
 : "${CACHE_TYPE_K:=q8_0}"
 : "${CACHE_TYPE_V:=q8_0}"
 : "${N_GPU_LAYERS:=999}"
 : "${BATCH_SIZE:=4096}"
 : "${UBATCH_SIZE:=1024}"
 : "${VERBOSITY:=2}"
 : "${CPU_MOE:=1}"
 : "${TEMPERATURE:=1.0}"
 : "${TOP_K:=20}"
 : "${TOP_P:=0.95}"
 : "${MIN_P:=0.0}"
 : "${THREADS:=7}"
 : "${THREADS_HTTP:=8}"
 MODEL="$MODEL_DIR/$MODEL_FILE"
 if [[ ! -f "$MODEL" ]]; then
    echo "missing required file: $MODEL" >&2
    exit 1
 fi
 if [[ ! -x "$LLAMA_SERVER" ]]; then
    echo "missing llama-server binary: $LLAMA_SERVER" >&2
    exit 1
 fi
 exec "$LLAMA_SERVER" \
    --model "$MODEL" \
    $([ "$CPU_MOE" = "1" ] && printf -- "--cpu-moe") \
    --cache-type-k "$CACHE_TYPE_K" \
    --cache-type-v "$CACHE_TYPE_V" \
    --ctx-size "$CTX_SIZE" \
    --parallel "$PARALLEL" \
    --gpu-layers "$N_GPU_LAYERS" \
    --batch-size "$BATCH_SIZE" \
    --ubatch-size "$UBATCH_SIZE" \
    --temp "$TEMPERATURE" \
    --top-k "$TOP_K" \
    --top-p "$TOP_P" \
    --min-p "$MIN_P" \
    --threads "$THREADS" \
    --threads-http "$THREADS_HTTP" \
    --alias "$ALIAS" \
    --host "$BIND_ADDR" \
    --port "$LISTEN_PORT"
@@ -1,6 +0,0 @@
 #!/usr/bin/env bash
 # Repository pre-commit hook: run only tests selected by staged paths.
 set -euo pipefail
 REPO_ROOT="$(git rev-parse --show-toplevel)"
 exec "$REPO_ROOT/scripts/test-changed.sh"
@@ -1,40 +1,6 @@
 # Local environment files — never commit real values
 .env
 *.env
 !.env.example
 # LLM fleet tunables (paths and numbers, never credentials)
 !scripts/*.env
 # LLM fleet runtime state (slot save/restore files written by the server)
 scripts/slot-state/
 # Playwright e2e: local creds, browser storage states, node artifacts
 tests/e2e/.env.local
 tests/e2e/.auth/
 tests/e2e/node_modules/
 tests/e2e/test-results/
 tests/e2e/playwright-report/
 # pi subagent artifacts
 .pi-subagents/
 # doc-audit scratch (transient, synthesized into serverAppConfigs-docs/ then trashed)
 .docaudit/
 # Agent trash
 .AGENTTRASH/
 # Python
 __pycache__/
 *.pyc
 # Virtual environments
 .venv/
 # Obsidian editor state
 .obsidian/
 # Editor/OS noise
 *.swp
 .DS_Store
 third_party/
@@ -1,827 +1,103 @@
-# AGENTS.md — serverAppConfigs Agent Rules
+# vLLM Scripts — Agent Instructions
-## Overview
+## Project Overview
-
+Scripts for running large language models via vLLM on two local machines:
-This repository is the single source of truth for the two homelab servers and
+- **dankserver**: NVIDIA GPU + System RAM (frank-default-small)
-the FrankBook-Pro Mac mini. It tracks the Podman container stacks and the LLM
+- **dankclaw**: AMD Strix Halo Shared Memory (frank-default-large)
 inference fleet. One repo, one checkout per host, one deploy script. All
 services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
 merge absorbed the former `vllmScripts` repo, history preserved).
 - **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
  Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
  via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
  Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
  container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
  RTX 3090). Caddy on
  Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
  (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
  its podman machine; all egress rides the Gluetun WireGuard netns. Deployed
  via the `frankbook-pro` handler.
 ## Fleet topology (ACTIVE)
 | Target | Service | Runtime | Payload | Port |
 |--------|---------|---------|---------|------|
 | Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
 | Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
 | Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
 Payload identity, shape, eras, laws and rollback live in ONE page each:
 [[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
 is the authority for every number; a page that disagrees with it is the bug.
 A retired payload is a row in that page's **Eras** table — there is no page
 per attempt and no separate swap-history file.
 ## Repository Layout
 - `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
 - `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
 - `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
 - `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
  scripts, systemd units, env tunables, bench harnesses
 - `old/` — retired LLM scripts (reference only)
 - `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
  The managed skill library is distilled there one page per skill-cluster:
  `LLM/index.md` → *Cross-Cutting References*, plus `Operations/index.md`,
  `Agent/index.md`, and the Infrastructure/Dankclaw/FrankBook-Pro service
  pages. There is no `log.md`: the change diary was destroyed 2026-10-19
 - `tests/` — bats suite + Playwright e2e
 ## Git Workflow — trunk-based (fleet law, aligned 2026-10-18)
 All edits flow through git. Never edit files on a remote machine directly.
 `main` is the only long-lived branch: no `develop`, no release branches, no
 back-merges. Frank is the only dev; this repo and `dank/command-center` follow
 the same regime.
 1. **Branch fresh** — `git fetch`, then branch off the current `origin/main`
   (`<type>/<slug>`, type ∈ feat|fix|docs|chore). If main moves under you,
   merge it into your branch — never rebase.
 2. **Source** the live config from the target machine
 3. **Commit** the fetched version here (redact secrets into `.env`)
 4. **Edit** locally
 5. **Test** — `./tests/run.sh` must pass
 6. **Push the branch and open a PR against `main`** — origin is Gitea on
   Dankclaw. Merge with `--no-ff`; **never merge a PR without an explicit,
   current OK from Frank.** **The moment the PR is open, open it in Frank's
   browser** (`xdg-open <PR-URL>` — a Firefox tab on the workstation); never
   make him hunt for it in Gitea. Releases: tag `vX.Y.Z` straight on `main`.
 7. **Pull** on the target (`deploy.sh` does this)
 8. **Deploy** — `scripts/deploy.sh` plans and dispatches only changed services
 **Attribution footer (issues and PRs)** — every issue or PR body you open MUST end with:
 ## Directory Structure
 ```
---
+vllmScripts/
-cwd: <working directory> · session: <omp session id>
+├── AGENTS.md          # This file — agent instructions
 ├── README.md          # Human-readable project overview
 ├── deploy.sh          # Automated deployment script
 ├── scripts/           # Executable vLLM launch scripts
 │   ├── validate-vllm.sh  # Validation tool
 │   └── *.sh             # Inference server scripts
 ├── tests/             # Test suite
 │   └── test_validator.sh
 ├── docs/              # Knowledge base
 │   ├── README.md
 │   └── *.md           # Research notes
 └── failedAttempts/    # Archived scripts (for reference)
 ```
-Session id from `/status`; `cwd` is that session's own working directory — not the build checkout, when they differ (note the build worktree in the body instead). Machine-made artifacts stay attributable to the box and session that spawned them.
+## Running Tests
 ```bash
 # Test validator on dankserver
 bash tests/test_validator.sh dankserver
 # Test validator on dankclaw
 bash tests/test_validator.sh dankclaw
 # Run all tests
 bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw
 ```
 ## Deployment
 ```bash
 # Deploy all scripts to all targets
 bash deploy.sh all
-```
+# Deploy specific script to dankserver
-./scripts/deploy.sh dankclaw
+bash deploy.sh dankserver vllm-qwen36.sh
-./scripts/deploy.sh dankserver
+
-./scripts/deploy.sh dankclaw --service searxng
+# Deploy specific script to dankclaw
-./scripts/deploy.sh dankserver --service frank-archon
+bash deploy.sh dankclaw vllm-thinkingcap.sh
 ./scripts/deploy.sh dankclaw --dry-run
 ```
-The planner maps changed paths to services; the remote runner restarts only
+## Git Workflow
 the selected service and records its revision outside the checkout.
 Documentation-only changes are a no-op. LLM services are selected only by
 their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
 by its unit/env/serve trio in `scripts/` — so
 unrelated
 container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
 runs on a host without passwordless sudo; the value never enters the repo.
-**`deploy.sh <target>` with no `--service` is a fleet-wide DRAIN, not a sync.**
+### Branching Strategy
-It also walks every per-service revision marker under the deploy-state dir and
+- `main`: Stable, working scripts only
-restarts each service whose marker is behind HEAD *and* whose paths changed
+- `dankserver/*`: Server-specific development
-since that marker — so a backlog accumulated by service-scoped deploys fires
+  - `dankserver/qwen36-awq`: Qwen3.6 AWQ experiments
-all at once, and one failing health gate aborts the run with every later
+  - `dankserver/*`: Feature branches for dankserver
-service left pending. Name the service unless a drain is what you want
+- `dankclaw/*`: Claw-specific development
-(`./scripts/deploy.sh dankserver --service frank-archon`).
+  - `dankclaw/*`: Feature branches for dankclaw
-### Deployment completeness (MANDATORY)
+### Worktree Pattern
 For parallel development:
 ```bash
 # Create worktree for feature
 git worktree add ../vllm-feature feature/qwen36-awq
-A deploy is **not complete** when `deploy.sh` exits green. It requires:
+# When done, merge back
-
+git worktree remove ../vllm-feature
 1. `./tests/run.sh` — all seams green, and
 2. the Playwright e2e suite (`tests/e2e/run.sh`) — runs **automatically** as
   the `[3/3]` gate at the end of `deploy.sh` against the LIVE fleet; a
   behavior-affecting deploy that fails it exits `DEPLOY INCOMPLETE`. Bypass
   only with `--skip-e2e` in an emergency. The gate runs the conditional
   `frankbook` Playwright project (the `*arr`/qBittorrent walk,
   `tests/e2e/specs/19-frankbook.spec.ts`) ONLY when a frankbook blast-radius
   path changed since origin/main — that spec, `frankbook-pro/**`,
   `shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the
   arr vhosts — so a sleeping FrankBook-Pro never fails an unrelated deploy.
   The default project set excludes it everywhere else (`E2E_INCLUDE_FRANKBOOK=1`
   opts in; see `serverAppConfigs-docs/Architecture/E2E Suite.md`).
 ```mermaid
 graph LR
    A[Edit locally] --> B[Tests pass] --> C[Commit + push]
    C --> D[deploy.sh plan] --> E[Target pulls ff-only]
    E --> F[Dispatch selected service] --> G[Per-service health]
    G --> H[Playwright e2e gate] --> I{green?}
    I -->|yes| J[Deploy complete]
    I -->|no| K[DEPLOY INCOMPLETE]
 ```
-## Boot Persistence & Self-Healing (MANDATORY)
+### Commit Guidelines
-
+- Use descriptive commit messages: `git commit -m "Add AWQ DFlash script"`
-Every deployed, working Podman container MUST have a self-healing,
+- Commit before deploying: `git add -A && git commit -m "Update for deployment"`
-boot-persistent Quadlet unit. A container that runs only from `compose.yaml`
+- Push to remote: `git push origin <branch>`
-or a manual `podman run` dies silently at reboot — rootless Podman does not
+
-auto-start bare containers, and `restart: unless-stopped` does not survive the
+### Deployment Flow
-explicit stop that shutdown performs. Never report such a service as "up".
+1. Make changes locally
-
+2. Run tests: `bash tests/test_validator.sh dankserver`
-Unit requirements:
+3. Commit: `git add -A && git commit -m "Description"`
-
+4. Deploy: `bash deploy.sh dankserver`
- **Boot persistence** — `[Install] WantedBy=default.target`, installed to
+5. Verify: `bash scripts/validate-vllm.sh -t dankserver`
-  `~/.config/containers/systemd/<name>.container`, enabled in the user
+
-  manager, with linger on for the deploy user (`loginctl show-user dank -p
+## Remote Servers
-  Linger` → `Linger=yes`).
+
- **Self-healing** — `Restart=always` (or `on-failure`) with a short
+### dankserver
-  `RestartSec`; quadlet health check where the service has one.
+- SSH: `dank@dankserver`
- **Repo provenance** — the `.container` file lives in this repo
+- Scripts: `~/vllm/`
-  (`<host>/<svc>/quadlet/`), is installed by the deployer, and is covered by
+- Service: `vllm-awq-qwen36.service`
-  the quadlet test seam. The host-side unit is generated state; the repo file
+- Model: frank-default-small (AWQ 4bit)
-  is the source of truth.
+- Port: 34401
-
+- Status: Disabled (pending bug fixes)
-Before declaring a service deployed: `systemctl --user is-enabled <name>` →
+
-`enabled`, `is-active` → `active`, and the container answers its health check.
+### dankclaw
-A running container with no unit is a gap: write the unit, test it, deploy it.
+- SSH: `dank@dankclaw`
-
+- Scripts: `~/vllm/`
-This rule exists because reboots found gaps twice (2026-08-26: Caddy as a
+- Service: `vllm-thinkingcap.service`
-bare compose container never returned; 2026-09-11: five observability
+- Model: frank-default-large (FP8)
-services had no units). The observability services now have units under
+- Port: 3090
-`dankclaw/observability/quadlet/` and their compose definitions were removed —
+- Status: Running (you are the deployed inference)
-one owner per container name. The stack is **configured and delivering as of
+
-2026-09-17**: a Grafana datasource and the `llm-fleet` dashboard are
+## Troubleshooting
-provisioned from `shared/grafana/provisioning/`, `shared/prometheus/rules/
+- Service failed: `systemctl status <service>`
-llm-fleet.yml` loads 5 alert rules + 1 recording rule, and firing rules are
+- Container stuck: `docker stop <name> && docker rm <name>`
-delivered: Prometheus → Alertmanager (`dankclaw/observability/quadlet/
+- OOM error: Reduce `--max-model-len` or increase `--cpu-offload-gb`
-alertmanager.container`, loopback :9093) → host-side Signal relay
+- Prefix caching conflict: Remove `--enable-prefix-caching` from script
 (`scripts/signal-relay.py`, user unit, 192.168.0.143:18093 — the host LAN
 IP, the one bind a container can reach as `host.containers.internal`;
 iptables admits loopback, the host and the podman bridge subnet only) → signal-cli account 318311.
 Prometheus targets read down for structural reasons (HTML-scraped "probes",
 Gitea metrics off); the remedy is a blackbox exporter. `frank-zealot` was down for
 a different reason until 2026-10-18: llama.cpp will not serve `/metrics` at all
 without `--metrics`. See
 `serverAppConfigs-docs/Infrastructure/Observability.md`.
 ## Testing
 Tests are mandatory before commit (`bats` required on the dev machine):
 ```
 ./tests/run.sh
 ```
 Seams: compose validity, env consistency (no leaked secrets), Caddyfile
 validity, quadlet units, docs completeness **including vault link integrity**,
 script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
 deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
 and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
 no project, because such a spec silently never runs.
 See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
 ## Secret Rules
 - Never commit real secrets. The live Gitea DB password and sudo password are
  test-enforced leak checks.
 - `.env` files are git-ignored; commit only `.env.example` with placeholders.
  Exception: `scripts/*.env` LLM tunables (paths and numbers, never
  credentials).
 - Do not document passwords anywhere in the repo. SSH keys are the auth
  mechanism; `FRANK_SUDO_PW` covers headless sudo and never enters the repo.
 - **A token passed as `curl -H "Authorization: Bearer …"` is world-readable.**
  It lands in `/proc/<pid>/cmdline`, so any `ps`/`pgrep -a` on the host prints
  it — that leaked an HF token into a session transcript on 2026-09-16, while
  merely checking whether a download was still running. Pass secrets out of
  band (`curl -K <mode-600 file>` carrying the header, or `HF_TOKEN` in the
  environment for `hf`/`hf_transfer`) and never `pgrep -a` a fetch command that
  carries an auth header.
 ## External Exposure Policy (MANDATORY)
 Every Caddy-proxied service is either behind the auth chain (Anubis :8923 →
 Authentik embedded outpost) or explicitly internal-only — no public DNS name,
 no vhost. New services default to internal-only until guarded.
 When guarding a new vhost — provider first, then route:
 1. Create the outpost proxy provider FIRST (`external_host` = exact vhost URL
   with trailing slash; `internal_host = http://caddy:8017`). If the vhost
   flips first, authenticated traffic hits the outpost error page.
 2. Flip the vhost to `reverse_proxy anubis:8923` with
   `header_up X-Real-Ip {remote_host}`, and add the post-auth
   `@<svc> host <vhost>` handler on the `http://:8017` listener routing to
   the real upstream.
 3. Anubis guard set, Caddy TLS edge, and outpost provider list change
   together in one deploy.
 4. Service-to-service traffic targets internal addresses (`loki:3100`), never
   the public hostname — non-browser clients cannot solve challenges.
 Exempt categories (document the reason in the vhost comment): the IdP itself
 (`auth.`), git smart-HTTP endpoints, services with their own complete auth.
 ## Deletion & Trash Policy (MANDATORY)
 Never `rm`. Move deletions into trash so they stay recoverable:
 1. In-repo files → `.AGENTTRASH/` beside the item's directory.
 2. Elsewhere → dated staging dir `~/.AGENTTRASH/YYYYMMDD-topic`.
 3. List staged items and sizes; wait for explicit user approval to purge.
 `rm`, `rm -rf`, and `git clean -f` are forbidden without explicit permission.
 `git * --no-verify` is forbidden. The trash is a temporary hold, not an
 archive.
 ## Model Swaps (fleet law, 2026-10-18)
 **Swap the model. Do not instrument it.** A payload change is: edit the trio
 (env / serve script / unit) → `scripts/deploy.sh <host> --service <svc>` →
 `/health` answers → one chat turn, one tool call, and one vision probe on an
 endpoint that serves images. That is the entire procedure.
 - **No benchmarking unless Frank orders it.** No pre-swap baseline battery,
  no A/B arms, no decode/prefill ladders, no acceptance tables, no numbers
  written into any page afterwards. Data exists only when he asks for it,
  and then only in the file he names.
 - **It doesn't work → roll back.** Revert the commit, redeploy, done. No
  post-mortem page, no investigation log, no trial-drop-in forensic tour. If
  the fault is not obvious, one line in the endpoint page's *Laws* section is
  the whole record.
 - What still gates a change: `./tests/run.sh` before commit, the deployer's
  per-service health gate, and the Playwright e2e gate. Those are the fleet's
  evidence. A bench is not evidence of correctness and buys nothing a swap
  needs.
 ### When a bench IS ordered
 **STOP the production service first** — these servers serve real users, and a
 bench beside a live service starves the GPU and corrupts the result. That
 covers **any GPU experiment, not just benchmarks**, including a bare
 2-rank RCCL/`torch.distributed` probe: on 2026-09-16 one was run against a
 live service "because it only needs 200 MB of VRAM" and the host went hard
 down ~20 minutes later (no ARP reply, 100 % ICMP loss, power cycle required;
 a hung AQL dispatch wedging the driver is the plausible mechanism). Telling
 an off machine from a wedged one: `ip neigh show <ip>` from a peer — FAILED
 means off or NIC dead; a merely wedged kernel usually still answers ARP.
 1. Stop and mask (self-healing would restart it):
   `sudo systemctl stop frank-dragoon frank-zealot` on Dankclaw,
   `sudo systemctl stop frank-archon` on Dankserver, then `mask` each.
   `mask` FAILS on a relinked unit (the file in `/etc/systemd/system/` is
   itself a symlink into the checkout); stop alone is enough for the window,
   and the deploy restores the unit — re-deploy after benching rather than
   unmasking.
 2. Bench against the service itself. No scratch ports on an LLM host — one
   process owns the card. Use the runtime's own harness (`llama-bench`; a
   live llama.cpp server also reports honest `timings` per request, including
   `draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build
   10777 that object only reaches you on **`/v1/chat/completions`**: a greedy
   `ignore_eos` run on the raw `/completion` path there answers **500 "The
   model produced output that does not match the expected Content-only
   format"** (measured 2026-10-18). halogen has no `/completion` path at all,
   so its rates are wall-clock around `/v1/chat/completions` plus the boot
   ledger. Generate ≥128 tokens: at 8, `inter_token` is first-token noise and
   run-to-run spread swamps every real difference.
 3. Unmask, start, verify `curl http://192.168.0.<host>:7770/health`.
   Retire every bench drop-in the same session
   (`systemctl show <svc> -p DropInPaths` must come back empty) and write the
   numbers into the file Frank named — `/tmp` and the journal are not records.
 Instrument traps that survive from the era when this was routine:
 `gufo bench` cannot rank speculative decode and **inverts the ranking** (its
 corpus reports `acceptance=0.933 ± 0.000` with an identical trace sha256
 across draft widths — the same predictable text every run); llama.cpp's
 `Server-Timing` carries the unit in the **name** half (`ttft;dur=402.485`),
 so splitting on `=` keys the dict `ttft;dur` and every lookup silently
 returns nothing; halogen 0.16.2 answers `/completion`, `/ping`, `/props` and
 `/tokenize` with 404 and sends no `Server-Timing`; and the first request
 after a boot is a warmup artifact (measured 2.6 t/s prefill then 1,258 t/s on
 the next), so never report request #1.
 **CRITICAL:** never `systemctl disable` a system unit on these hosts — it
 deletes the unit file from `/etc/systemd/system/`, not just the symlink.
 `mask` symlinks to `/dev/null` and is safe/reversible.
 ## Documentation Policy (fleet law, 2026-10-19)
 **Documentation exists for point of use.** A page earns its bytes only if it
 helps whoever performs the next task — the shape to deploy, the flag that must
 be passed, the failure signature to recognise, the rollback to run. It is not a
 lab notebook and not a ledger. **Losing information is an acceptable, usually
 correct outcome.** Never let an orphan census ("this is the only copy of that
 fact") decide what to keep: sole custody of a useless fact is still useless.
 - **The purpose test — before you write, and again before you keep:** would the
  reader act differently because of this line? No → delete it.
 - **Measurements are disposable.** A rate or throughput figure (t/s, tok/s, ms,
  acceptance %, a bench ladder, a measured MiB) for a payload, runtime, or
  layout that is not running now and not being compared now is destroyed, not
  filed — not in an Eras row, not on a technique page, not "preserved for
  history". AI moves faster than this vault: the number is wrong before anyone
  reads it, and a wrong number in a runbook gets believed. Keep the **law** the
  measurement taught; drop the measurement.
 - **One fact, one owner**, every other file links with `[[Page]]`. The env file
  beside the serve script is the authority for every live number; the endpoint
  page is the authority for the shape and the wire contract.
 - **One page per endpoint:** current shape, **Eras** (`window | payload |
  runtime | replaced because | rollback` — the rollback is a commit, not a
  number), the **Laws**, the wire contract, the files that own it. A swap edits
  the shape block and appends one row; the retired era's own page is
  **destroyed** to `.AGENTTRASH/`, never merged into anything.
 - **`log.md` is retired** (2026-10-19: 2,380 lines destroyed). Git holds the
  chronology. A causal law belongs to the page that owns the thing it governs,
  or to *Key Lessons Learned* below if it is cross-cutting — never to a diary.
 - **Index pages are locators:** one row, one clause, no numbers.
 ## Model Config (harness)
 The harness reaches the fleet through `.omp/agent/models.yml` in the
 operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
 where the harness runs; there is no copy on the servers. Each provider is one
 HOST:PORT; each model `id` is the served alias. **Official model ids are
 the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
 since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
 `192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
 `192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
 `192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
 scripts, env files, containers, deploy services and runbooks to match —
 the three former service names are fully retired and exist nowhere in
 the repo; historical text in runbooks uses the current names. Verify a
 rename on `/v1/models`, never
 on a chat response: radiance echoes the **requested** model name, so a 200
 proves nothing about the served id (llama.cpp and halogen ignore it).
 Resolution rules: the default model resolves as a `defaultProvider` +
 `defaultModel` pair (the provider must serve the model); a subagent override
 uses a bare model `id`, which resolves only when unique across providers. Two
 ports on one host need two providers — the file cannot express "same host,
 different port".
 When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
 of 2026-10-06 **all three endpoints accept images** (`input: [text,
 image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
 (the v2 vision tower loads beside the checkpoint; the text-only halogen
 27B window closed 2026-10-06) — while
 `frank-archon` takes them natively (radiance runs the checkpoint's
 27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
 are the engine core; `--mm-max-patches` available),
 and `frank-zealot` through
 the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
 publisher's requirement for concurrent image requests (the card states the
 BF16 tower can crash `llama-server` when several image requests run at once,
 and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
 **200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
 2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
 deliberately below the model's native window; llama.cpp answers a longer prompt
 with a 400 naming the limit, never a truncation, so the harness must compact
 before it.
 Note also
 that
 the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
 parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
 models.yml sets `reasoningContentField` per endpoint to match.
 There is **no audio**
 input anywhere in the fleet: that capability left with the FLM NPU
 endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
 (neither the MiMo nor the Qwen3.5 tower carries audio).
 Each endpoint has ONE page in the vault: shape, Eras, Laws, wire contract,
 rollback. The numbers below are only what a consumer or a deploy gate needs;
 the env file beside the serve script is the authority, and a page that
 disagrees with it is the bug.
 ### frank-dragoon — halogen flash-server, Dankclaw iGPU, :7770
 - Owns: `scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
  `scripts/frank-dragoon.service`, container `frank-dragoon-halogen`, image
  digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`.
  Engine stdout (the boot ledger: weights, KV pool, clamp) goes to
  `/home/dank/logs/frank_dragoon.log`, NOT the journal.
 - Shape: `HALOGEN_CTX=200704` is the **per-request ceiling** (a longer prompt
  is a 400, never a truncation); `HALOGEN_KV_SLOTS=2` over ONE
  `HALOGEN_KV_POOL_POSITIONS=401408` pool — pool positions buy concurrency,
  the per-request window never moves; `HALOGEN_SPEC_ADAPT=0` keeps the MTP
  drafter always armed (the adaptive default disarms it and halves decode);
  server sampler 1.0/0.95/20.
 - Harness contract: `models.yml` `contextWindow: 155648` = ceiling − 45,056.
  The clamp is `min(45056, window − est − 4096)` and the chars/4 estimate
  under-counts, so a harness window equal to the ceiling makes every long
  request a guaranteed 400. Reasoning arrives in `reasoning_content`; the
  template rejects `reasoning_effort` outside `xhigh|medium|low`; images are
  OpenAI `image_url` parts (probe ≥256 px); `response_format` unsupported;
  `/completion`, `/ping`, `/props`, `/tokenize` are 404.
 - Page: [[frank-dragoon]].
 ### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
 - Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
  `scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY
  loader (the payload carries the fork's custom PQ2_0 tensor type plus
  `prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
  loader ignoring that metadata dequantizes wrong); payload
  `Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
  tower can crash the server with concurrent image requests) + the rank-1
  refusal-direction adapter applied at load by `--lora-scaled …:1`. The
  adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
  — and build 10777 is also the gate that lets it load: older forks refuse it
  with `unsupported prism.hadamard.version: 2`.
 - Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
  200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
  B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
  `SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
 - Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
  (`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
  with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
  reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
  metadata key, so it must reach argv. This endpoint is the fleet's
  compaction hub.
 - Page: [[frank-zealot]].
 ### frank-archon — radiance, two R9700s TP=2, Dankserver :7770
 - Owns: `scripts/frank-archon.env`, `scripts/serve-frank-archon.sh`,
  `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
  `scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
  `stilldeadcode/radiance@sha256:d590e300…` (radiance 1.8.1); container name = unit name.
 - Shape: TP=2 over `wht6`, `--max-num-seqs 2`, `--max-model-len 200000`,
  `expert_tiered` placement, 44,928 MiB pinned host pool per rank, `fp8` KV,
  MTP depth 2, MNBT 2048, `--busy-prefill-chunk auto`, prefix cache 8 GiB host
  tier only, n-gram table pinned to RAM. The per-card weights/KV split is derived
  by the engine's budget resolver from `--gpu-headroom-mib`, the cache bound and
  the host pool; the `--vram-weights-mib`/`--vram-kv-mib` pair and the prefix disk
  tier are gone (removed at 1.3.0 / 1.8.1) and naming either is a boot refusal,
  not a no-op.
 - Harness/gate contract: `models.yml` `contextWindow` MUST quote 200,000
  (over-length is refused, never truncated); the ~147 GiB pinned-RAM ledger
  is why this is a root unit, and `ttm-preflight.sh` raises
  `ttm.pages_limit` itself because the kernel resets that knob to half of RAM
  on every boot; `/health` binds only after the full load, so a bare health
  gate is honest.
 - Page: [[frank-archon]].
 ## Key Lessons Learned
 Durable operational lessons from the fleet's history. Full detail in the
 linked runbooks.
 **PCIe / multi-GPU**
 - **RCCL collectives need PCIe atomics on every participating GPU, and the failure never names
  atomics.** The kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch them on
  a device without atomics and the error surfaces as `hipErrorIllegalState` from `hipLaunchKernel`,
  not as anything that names atomics. The value is consumed at driver PROBE time, so writing the
  AtomicOp Requester bit with `setpci` at runtime changes nothing. Check the CHAIN: `lspci -vv -s
  <gpu>` and every upstream bridge must show `AtomicOpsCap: Routing+` (a `Routing-` bridge is a
  hardware wall), and the endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
  6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
  chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
  nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
 - **When a GPU collective dies, read the runtime, not the framework.** The message that identified
  the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
  `AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
  failing HIP call with its return code. [[PCIe Link and Atomics]]
 **ROCm / GPU**
 - **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a trap.** With a missing
  `DST`, cp creates `DST` as a copy of `SRC` and drops one directory level. rocBLAS and hipBLASLt
  look for their device code at a FIXED depth (`../lib/rocblas/library`,
  `../lib/hipblaslt/library/<arch>`), so the flattened copy fails at REQUEST time, not load time:
  the engine reads the weights, answers `/health` and dies on the first completion with
  `Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch : gfx1201`. Declare
  the destination, then copy the contents.
 - **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
  anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
  and pinning the PLE tensor with `-ot` was slower than letting it page. [[Llama Cpp Serving Sizing]]
 - **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
  with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
  `1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
  tunable these; check with
  `strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
 - **`GGML_CUDA_FA_ALL_QUANTS` is dead; do not build for it.** Verified on this host 2026-10-19:
  master's `ggml/cmake/common.cmake` warns it is deprecated in favour of `GGML_CUDA_FA_QUANTS`, the
  successor's DEFAULT pair list (`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16`) already compiles the
  quantized-KV kernels with the old flag OFF, and neither llama tree on this host references the old
  macro anywhere under `ggml/src/ggml-cuda/`. The old "~12× prefill collapse" belongs to the older
  source line. Prove a build's KV ability from its `CMakeCache.txt`. [[frank-zealot]]
 - **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
  `blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
  loader support.
 - **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
  Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
  `/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
  `free`/`dmidecode` show the CPU-side carve-out — intentional, do not change. [[Dankclaw System Specs]]
 **Load / crash diagnosis**
 - Model load is slow (minutes cold) and health returns 503 while it runs: do NOT kill the process,
  `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On halogen the FIRST request after a (re)start
  can go silent for ~45–60 s (`engine_unresponsive`, 503s) and then self-recovers — do not restart
  over it. [[frank-dragoon]]
 - Crash-loop procedure: (1) `journalctl -u <svc> -n 100` AND the kernel line via `journalctl -k` (a
  segfault at address 8 in `libamdhip64` is a NULL unref in HIP); (2) run the serve script, then the
  raw binary — a different outcome isolates the env file; (3) `gdb -batch -ex run -ex "bt 25"
  --args bash scripts/serve-<svc>.sh`, because llama-server buffers stdout and loses the log tail on
  SIGSEGV: trust gdb, not the log; (4) `amdgpu: SVM mapping failed` spam is a symptom, not a root
  cause. [[LLM Performance Diagnosis]]
 - Journal signatures: a client abort logs `W srv stop: cancel task, id_task = N` while a natural stop
  logs only `slot release` with `truncated = 0`. The server never preempts a running generation;
  excess tasks queue when slots are busy.
 **Host memory / swap**
 - **Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` of the old one**, or
  live pages must land in RAM that `MemAvailable` overstates. Dankclaw's 64 GiB `/swap.img`
  (fstab-driven) protects the co-tenants, not an endpoint with pinned weights — paging the KV pool
  is worse than the OOM it prevents. Host swap state is invisible to `deploy.sh` (no repo file):
  `swapon --show` before reasoning about headroom. [[Swap Configuration]]
 - **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults to half of RAM, so
  a manual raise vanished with a power failure, frank-archon's pinned-ledger preflight refused the
  start, and `Restart=always` re-ran that refusal 6,919 times in ~20 h — **a unit in
  `activating (auto-restart)` that exits 1 in 50 ms with no container and no engine log is a
  PREFLIGHT refusal, not an engine fault.** Repo law now: `scripts/ttm-preflight.sh` raises the knob
  to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only after a write the
  kernel rejected. Never prescribe a manual sysfs write as the recovery procedure for a service that
  can boot before a human reaches it. [[frank-archon]]
 - **Silent RAM corruption outranks every software theory.** frank-archon's long-conversation
  repetition/garbage was marginal DDR5 at 4800 MT/s on four DIMMs, fixed by dropping to 4600.
  Consumer non-ECC raises NO MCE for a silent flip, and a short in-band pattern check (60 s over
  8 GiB of 246 GiB) is not a stability proof — both instruments said "ruled out" while the RAM was
  the cause, and "4800 is the JEDEC floor" is a rating, not a margin guarantee. When the fault is
  data-shaped (NaN states, garbage tokens) and a software revert does not move it, price in the RAM
  before the runtime; a revert tested under the same marginal hardware is itself confounded.
  [[frank-archon]]
 **Runtime-specific guards**
 - **Rootless podman cannot RAISE rlimits, so `--ulimit nofile=1048576` dies at container CREATE**,
  not at runtime: runc reports `error setting rlimit type 7: operation not permitted` (type 7 =
  RLIMIT_NOFILE) when the requested hard limit exceeds the host process's ceiling (524288 on
  Dankclaw). Inherited-equal limits are fine, so `memlock=-1:-1` works iff the unit grants
  `LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
  any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
  before adding flags. [[frank-dragoon]]
 - **A GGUF's custom tensor type picks the runtime — read the histogram before promising a
  file to a loader.** A non-standard tensor type in the KV section makes stock loaders refuse
  the file at load, so a fork can be the only door rather than a preference. Counting tensor
  types takes one python read and settles a runtime argument in minutes; build identity matters
  as much as the fork, so pin the commit by SHA in any build script. [[GGUF Compatibility]]
 - **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
  (the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
  baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
  measures a crash instead of the absence of drafting. [[frank-zealot]]
 - **An orphaned scratch server taxes the whole host.** A bench child left alive by a client-side
  exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
  every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
  back before believing a number — and kill by pid, never `pkill -f llama-server`.
 **llama.cpp serving**
 - **MTP is a property of the FILE, not of the flags.** The `-mtp` export carries the head as
  `blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`); `--spec-type draft-mtp` sets
  `load_mtp` and the draft context runs on the MAIN model's weights (`shares_model`), so there
  is no second copy of a 12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
  tensors at all, and pointing `--model-draft` at that file loads it twice. [[frank-zealot]]
 - **What can be chained: ONE context-backed drafter plus any number of draftless ngram types.**
  `--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
  `ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
  `draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
  legal and it wins on prompt-echo traffic, but `draft-mtp,draft-dflash`
  dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
  chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
  hybrid loses, because adding the ngram layer logs
  `recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
  MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
  [[frank-dragoon]]
 - **A drafter's KV scales with the TARGET's context, and its window policy decides the price.**
  The draft context is built at `llama_n_ctx(ctx_tgt)` with no flag to cap it, so a drafter
  declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
  first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
  drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
  [[Llama Cpp Serving Sizing]]
 - **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
  compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
  wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
  cost against what the batch knobs leave for compute. Measure the arm, never the flag.
  [[frank-zealot]]
 - **A quant's scale overhead belongs in its byte rate; doubling a measured figure to "get the
  f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element (32 codes + one 2-byte scale per
  32-block), f16 is exactly 2.0 ⇒ the ratio is **1.882×**, not 2×. Derive byte rates from element
  count × block layout, then check them against the boot ledger — this repo carried an invented
  2× number for weeks. Current KV math: [[frank-zealot]]
 - **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is BACKEND-SPECIFIC — never
  carry it across hosts.** Its compute buffers land about TWICE as heavy on the card that holds
  the output layers and the drafter, so a split balanced at one batch is unbalanced at another:
  re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
  large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
  model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
  control that proves the knob, not the traffic, moved the number. The optimum is a property of
  the backend, so re-find it after any device or payload change. When a bench IS ordered, A/B it
  by restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
  port: one card holds one server.
 - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
  boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
  by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
  [[Llama Cpp Serving Sizing]]
 - **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
  matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
  only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
  costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
  too short and it proposes on ordinary prose fragments, costing verification without acceptance.
  [[Llama Cpp Serving Sizing]]
 - **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
  the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
  prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
  `contextWindow` MUST follow every reslice. Both slot directions cost or buy window, never
  pool; shorter slots decode slightly faster. Measured reslices: [[frank-zealot]].
 - **Price a VRAM budget from two boots, never from a formula — and an mmproj costs ~1.5× its file
  size** (the encoder's activation and compute buffers ride with it). Subtracting two booted rows
  isolates it exactly; extrapolating weights+KV+"a bit for buffers" is what leaves 1.1 GiB free
  when you meant 2.7. Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
  `PARALLEL × 4096`. **The projector is NOT billed in llama.cpp's own fit check**: the engine
  prints its worst case and fits WITHOUT it, so price a shape from settled `nvidia-smi` after a
  real boot — the projector, and any drafter context, lands after `/health` would already have
  answered (same class as the DSpark drafter OOM). Two-boot table: [[frank-zealot]].
 - **`images` and `image_url` are NOT interchangeable: only the content part reaches the model.**
  `--mmproj` loads, `usage.prompt_tokens` rises, and the model still invents an answer if the
  client used the legacy top-level `images` field on `/v1/chat/completions` — frank-zealot
  answered "Brown." for a solid-red PNG that way and "Red." through the OpenAI `image_url` part.
  The failure is silent because a hallucinated color is a plausible answer; probe with a solid
  color and expect an exact word. [[frank-zealot]]
 - **`--jinja` is what `tools` needs.** A subagent died with `500 tools param requires --jinja
  flag` on the ik_llama.cpp runtime that served frank-dragoon: every tool-using agent on that
  endpoint was broken and the failure read like a model fault. Mainline llama.cpp defaults
  `--jinja` on and the serve scripts pin it; halogen also accepts `tools` and returns
  `tool_calls`. Check tool support before routing a role at an endpoint.
 - **A model's own template can REJECT the harness's thinking level.** Qwen3.8's template accepts
  only `xhigh`, `medium` and `low` for `reasoning_effort` and raises on anything else — HTTP 500,
  not a clamp — so the harness's levels must be folded onto that set in `models.yml`
  (`compat.reasoningEffortMap`). A role whose level is unmapped fails on its FIRST request and
  the failure names Jinja, not the model: every subagent on `frank-zealot` died with
  `Unexpected reasoning effort high` until `high: xhigh` was added. Halogen accepts the wider
  set, so the two entries legitimately differ.
 - **Thinking tokens count against `max_tokens`, so a short-budget vision probe reads as
  blindness.** A probe at `max_tokens: 64` returned `content: ""` with `finish_reason: "length"`
  and half a thought in `reasoning_content` while the image HAD reached the model (322 prompt
  tokens vs 42 for the text alone). Budget ≥512 for any thinking model you probe for vision.
 - **The FIRST request after a boot is a warmup artifact**, and the 3× spread on one shape makes
  graph-capture cost read as a model fault. Benchmark the second request or later.
  [[frank-zealot]]
 - **A fork build hides its own dtype laws.** Vendor builds carry their own compile options, so
  prove a build's KV behaviour from the KV buffer line in the boot ledger and its
  `CMakeCache.txt` — never from the build flags you remember passing. [[frank-zealot]]
 - **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed**: without it
  the engine answers **501 Not Implemented** on `/metrics`, so the scrape fails while `/health`
  answers 200 and `LlmEndpointDown` fires beside a healthy service. halogen serves `/metrics`
  natively, so the asymmetry reads as "only the 3090 is broken". Diagnose with `lastError` from
  `/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot
  re-stamps `startsAt` on an alert that has fired for weeks. [[Observability]] [[frank-zealot]]
 - **A leftover bench drop-in silently hijacks the next deploy, and no health gate sees it.** The
  sanctioned bench-window method (a transient `Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`
  drop-in sourcing the repo env) outlives the bench unless retired: a leftover
  `frank-zealot.service.d/ptq1.conf` made the cutover deploy boot a **different model at a
  different shape** while `/health` answered 200, `/v1/models` echoed the alias (a flag, not a
  fact) and the Playwright gate passed 33/33. After deploying any recently-benched LLM service:
  `systemctl show <svc> -p DropInPaths` (must be empty), then
  `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger — and retire drop-ins
  to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`. [[frank-zealot]]
 - **A bench verdict that never reaches the repo is indistinguishable from a rogue arm.** That
  same arm served production for 24 minutes with real numbers (~48 t/s decode, 801 t/s prefill at
  141k) that lived only in `/tmp` env files and journald, so the next agent read the surviving
  drop-in as an unfinished experiment and reported a hijack. Write the measurement into the
  endpoint page and the arm script's header in the session that takes it — `/tmp` and the journal
  are not records.
 - **`llama-bench` separates a LIST with `/` and SWEEP values with `,`** — for both `-dev` and
  `-ts`. `-dev CUDA0,Vulkan1` is two *single-device* configurations, never one two-device run,
  and `-ts 0.85,0.15` is two sweep values each putting everything on device 0. Write
  `-dev A/B -ts x/y`. This artifact produced a wrong fleet conclusion ("CUDA + Vulkan cannot serve
  one model") from two bogus OOMs, and the bench table's own `dev` column was printing the truth
  the whole time. Any placement-dependent number needs an independent check: sample the other
  device's memory (`nvidia-smi`) while the run is loaded.
 **Ops**
 - Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
  Use `pkill -x <exact-name>`.
 - Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
  (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
 - **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
  `GroupAdd=keep-groups`, plus the udev rule that makes those nodes world-writable — user-unit
  spawns carry no supplementary groups) and must NOT pass `--cpu`: the gfx1151 ROCm wheels exist
  and a stale device-less container was found restarted 9,146 times. [[ComfyUI]]
 - SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun
  service. [[SearXNG]] [[Gluetun]]
 - **Dankclaw's kernel command line is a repo file.** It boots **`iommu=pt`** since 2026-10-18,
  tracked in `dankclaw/host/grub`, and moves ONLY through `scripts/set-dankclaw-bootline.sh`
  (diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by hand-editing
  `/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
  says identity mode costs nothing, so the XDNA2 NPU returned for free.
  [[dankclaw-iommu-translation-off]]
 - **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
  instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
  rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
  [[Quadlet Lifecycle]]
 - **A dead rootless podman pause process breaks ALL container management on the host, and
  `podman system migrate` panics in 5.7.0.** Every `podman` command fails with
  `invalid internal status`, every start fails binding the bridges
  (`failed to bind udp listener on 10.89.x.1:53`), every `Restart=always` quadlet loops — while
  container storage stays intact. What is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
  token) and the rootless netns; deleting the stale token is not enough. Repair: **reboot**
  (`/run/user` clears and the quadlets return on their own — check `Linger=yes` and
  `WantedBy=default.target` first). The 2026-09-16 occurrence had no OOM in `journalctl -k`:
  treat the pause process as host state that can die for its own reasons. [[Quadlet Lifecycle]]
 - **A container cannot reach a `Network=host` service at the host's LAN IP**: netavark DNATs
  *published* ports only, so the host bind and the `10.89.1.1` gateway are both refused from a
  bridge container while the published port answers. Use `host.containers.internal`
  (169.254.1.2). This 502'd `hermes.franksdank.cloud` until its Caddyfile upstream changed.
  [[Container Host Reachability]]
 - **A `*.gguf` filter misses mradermacher's payloads: large quants ship as `.partNofM`
  fragments** while the smaller rungs are single files, so filtering `endswith(".gguf")` makes a
  whole quant class look absent. Fetch the parts, verify each against the Hub's LFS sha256,
  concatenate in numeric order. [[Model Swap and Rollback]]
 - **A fetch script must `-f` and verify, or it reports success on an access error.**
  `curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
  repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
  seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
  declared size, and check the LFS sha256 — the live `scripts/download-frank-*.sh` scripts do
  all three; ad-hoc fetches tend not to. Gated repos 403 even with the host's HF token until
  that account accepts the gate. [[Model Swap and Rollback]]
 - **A verdict is a lead — the scout's and the tool's alike.** In the 2026-10-19
  documentation campaign six scouts worked the same pages and produced invented
  bullet IDs, files that do not exist, "duplications" grep disproved, line
  numbers past the end of the file they judged, and Eras tables with invented
  payloads. The mechanical tool was worse: `scripts/port-census.py` built its
  corpus from the directory containing the page under test, so each page was held
  by *itself* and ten era pages reported "0 sole-record lines" — which reads as
  "pure duplication, trash it". Nothing was cut on a verdict; every holder was
  re-grepped and every tool re-measured before it was believed.
 - **Never let a census decide what to keep.** That same campaign then concluded
  the era pages were irreplaceable because each held 14–63 sole-record lines —
  wrong in the opposite direction: those were throughput figures for payloads that
  no longer run. Point of use is the only ownership test, the operator applies it,
  and 53 pages plus the 2,380-line change diary were destroyed as a result. A tool
  can say where a fact lives; it cannot say whether anything needs it.
 ## Runbook Index
 One page per live endpoint; every payload swap it has ever run is a row in that
 page's **Eras** table, with the rollback path in the same row. There is no
 per-attempt page and no per-swap ledger anywhere else.
 | Service | Page |
 |---------|------|
 | `frank-archon` — radiance, 2× R9700 TP=2, Dankserver | `serverAppConfigs-docs/LLM/frank-archon.md` |
 | `frank-dragoon` — halogen flash-server, Dankclaw iGPU | `serverAppConfigs-docs/LLM/frank-dragoon.md` |
 | `frank-zealot` — llama.cpp (PrismML fork), RTX 3090 | `serverAppConfigs-docs/LLM/frank-zealot.md` |
 | `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
 | Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
 There is no page for a retired endpoint and no change diary. A payload swap
 appends one row to the live endpoint's **Eras** table; when the endpoint itself
 dies, its page is destroyed. Rollback is a commit, not a document.
 `serverAppConfigs-docs/index.md` is the locator for every page that survives;
 retired scripts live in `old/` and `.AGENTTRASH/`.
 ## Why
 Direct edits on the remote drift from version control, and future agents
 won't know what changed. Git is the single source of truth. The two workloads
 share two hosts, one Caddy, and one Gitea; one repo keeps them in one
 history.
 ## Summary
 | Context | Action |
 |---------|--------|
 | Editing a compose/Caddyfile/serve script | **Through git** — fetch, commit, edit, test, push, pull, deploy. Never edit on the remote. |
 | Config exists only on remote | **Pull it into the repo first** — commit the fetched version (redacted) before editing. |
 | Unstaged local changes | **Commit or discard** — check `git status` before starting work; ask if unsure. |
 | Deploying a committed change | **`scripts/deploy.sh <target>`** — pushes, pulls ff-only, plans, dispatches, health-checks. |
 | Container without a Quadlet unit | **Give it one** — boot-persistent, self-healing `.container` via git. |
 | Changing a model/payload | **Swap it** — edit the trio, deploy, `health` + one chat/tool/vision probe. No benches unless Frank orders one; if it misbehaves, revert. |
 | Writing a doc | **One owner per fact** — edit the endpoint/service page; everywhere else links. Never restate a number. |
 | Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |
@@ -1,74 +1,105 @@
-# Server App Configs
+# vLLM Scripts
-### AI Disclosures:
+Scripts for running large language models via vLLM on two local machines.
 - Slop rating: 2/5 - Actively using, actively de-slopping, **repository mostly human owned**, lacks formal process.
 - Completeness: Unfinished, planning planned, no plan yet
 - Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
 - Local model: Qwen 3.8 Flash Next
 ### slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
 ### slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
 The single repository for the two homelab servers: the Podman container
 stacks **and** the LLM inference fleet. One repo, one checkout per host,
 one deploy script.
 > 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
 > into this repo with its full git history. See AGENTS.md for the merge
 > note.
 ## Targets
-| Host | User | IP | Podman stack | LLM service | LLM port |
+| Host | User | Codename | Architecture | Service Status |
-|------|------|----|--------------|-------------|----------|
+|------|------|----------|-------------|----------------|
-| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
+| dankserver | dank | DankServer | NVIDIA GPU + System RAM | `vllm-awq-qwen36.service` (failed) |
-| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
+| dankclaw | dank | DankClaw | AMD Strix Halo (Shared Memory) | `vllm-thinkingcap.service` (failed) |
-## Active Services
+## File Locations
-| Service | Host | Hostname | Port |
+### Local (vllmScripts/)
-|---------|------|----------|------|
+- `scripts/` — Active inference scripts
-| RomM | dankserver | romm.dankserver.local | 8085 |
+- `tests/` — Test suite
-| Copyparty | dankserver | files.dankserver.local | 3923 |
+- `docs/` — Knowledge base
-| Calibre-Web | dankserver | books.dankserver.local | 8083 |
+- `failedAttempts/` — Archived scripts
-| frank-medium | dankserver | — (bare endpoint) | 7770 |
+
-| SearXNG | dankclaw | search.dankclaw.local | 8081 |
+### Remote (dankserver)
-| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
+- `~/vllm/` — Deployed inference scripts
-| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
+- `~/vllm/failedAttempts/` — Archived scripts
-| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
+- `~/vLLM/` — Legacy project directory (vllm-qwen36.sh)
-| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
+
-| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
+### Remote (dankclaw)
-| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
+- `~/vllm/` — vLLM source code (not inference scripts)
-| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
+- `~/vLLM/` — Inference scripts (if deployed)
 ## Models
 | Model | Target | Format |
 |-------|--------|--------|
 | Qwen3.6 35B A3B FP8 | dankserver | FP8 |
 | Qwen3.6 27B FP8 | dankclaw | FP8 |
 | Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ 4bit |
 | Qwen3.6 35B A3B DFlash | dankserver | Draft model |
 ## Structure
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the Dankserver stack
+- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, and deployment notes
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the Dankclaw stack
+- `scripts/` — Executable vLLM launch scripts
- `shared/` — homepage, prometheus config, deploy helpers
+- `vllm-qwen36.sh` — FP8 inference script for dankserver
- `scripts/` — `deploy.sh` and `deploy-remote.sh` (service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnesses
+- `vllm-awq-qwen36-dflash.sh` — AWQ 4bit with DFlash speculative decoding (docker)
- `old/` — retired PP/NPU-era LLM scripts (reference only)
+- `vllm-awq-qwen36-dflash-ampere.sh` — AWQ 4bit with DFlash speculative decoding (Ampere-optimized, systemd)
 - `serverAppConfigs-docs/` — project knowledge base (Obsidian vault, wikilinks). `serverAppConfigs-docs/LLM/` is the LLM fleet
 - `tests/` — bats test suite (see [[Testing Policy]])
 ## Deployment
 See [serverAppConfigs-docs/Architecture/Deployment Flow.md](serverAppConfigs-docs/Architecture/Deployment%20Flow.md)
 and [serverAppConfigs-docs/LLM/deployment.md](serverAppConfigs-docs/LLM/deployment.md). Run
 `./scripts/deploy.sh <dankclaw|dankserver>` after committing changes. It
 plans and deploys only the changed services. Use `--service NAME` for an
 explicit single-service retry; inference is restarted only when its own files
 (or the shared chat template) changed.
 ## Knowledge Base
-The `serverAppConfigs-docs/` directory is an Obsidian vault. Start at
+See [[docs/README.md]] for the project knowledge base index.
 [serverAppConfigs-docs/index.md](serverAppConfigs-docs/index.md). The LLM fleet knowledge base lives at
 [serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
-## Testing
+## Lessons Learned
-Run `./tests/run.sh` before every push. See
+### AWQ + DFlash on Ampere Hardware (2026-08-08)
-[serverAppConfigs-docs/Architecture/Testing Policy.md](serverAppConfigs-docs/Architecture/Testing%20Policy.md).
+
 **What worked:**
 - `cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit` loads successfully on Ampere GPUs
 - AWQ Marlin kernel engaged automatically for weight conversion
 - DFlash draft model `z-lab/Qwen3.6-35B-A3B-DFlash` detected as EAGLE model
 - torch.compile with cudagraphs works with `--max-num-batched-tokens 1536`
 **What didn't work:**
 - `--enable-prefix-caching` conflicts with Mamba cache "align" mode — causes `AssertionError` in `unify_kv_cache_spec_page_size`
 - `--max-model-len 262144` causes OOM during AWQ→Marlin weight conversion on 24GB GPU
 - Nightly vLLM `--speculative-config` requires `num_speculative_tokens` when model is provided
 - `--speculative-model` flag removed in nightly; use `--speculative-config` JSON instead
 **Working configuration:**
 - `--max-model-len 32768` (32k context)
 - `--cpu-offload-gb 40` with `--cpu-offload-params "experts"`
 - `--gpu-memory-utilization 0.95`
 - `--performance-mode throughput`
 - `--enable-chunked-prefill`
 - `--max-num-batched-tokens 1536`
 - Thread binding: `OMP_NUM_THREADS=8`, `VLLM_CPU_OMP_THREADS_BIND=8`
 **Performance observations (no speculative decoding):**
 | Streams | Prompt Speed | Completion Speed | Total Time |
 |---------|-------------|-----------------|------------|
 | 1 | 4.27 tok/s | 21.42 tok/s | 11,951ms |
 | 2 | 2.44 tok/s | 12.25 tok/s | 20,891ms |
 | 3 | 1.35 tok/s | 6.76 tok/s | 37,849ms |
 | 4 | 1.43 tok/s | 7.19 tok/s | 35,606ms |
 Note: Speeds decrease with more streams due to KV cache contention.
 **Still needs work:**
 - DFlash speculative decoding integration (KV cache assertion bug in nightly)
 - Full context length support (32k is temporary workaround for OOM)
 ## Validation
 Use `scripts/validate-vllm.sh` to test inference servers:
 ```bash
 # Test dankserver (default)
 bash scripts/validate-vllm.sh -t dankserver
 # Test dankclaw
 bash scripts/validate-vllm.sh -t dankclaw
 # Custom host/port
 bash scripts/validate-vllm.sh -H 192.168.1.100 -P 8001 -M my-model
 # Run tests
 bash tests/test_validator.sh dankserver
 ```
@@ -1,26 +0,0 @@
 #!/usr/bin/env bash
 # autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
 #
 # The service runs on dankserver (192.168.0.235:7770), the two R9700s live
 # there, and the benchmark must own those cards, so the whole stop -> boot
 # scratch -> bench -> restore sequence runs ON dankserver (see
 # scripts/autoresearch-frank-archon.sh). This wrapper:
 #   1. copies the candidate config (the repo's scripts/frank-archon.env, the
 #      file the autoresearch loop edits) and the bench script to dankserver,
 #   2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
 #      exit code back here verbatim.
 # It deliberately does NOT push to origin: the scp dirties dankserver's
 # working tree, and the loop's uncommitted candidate config must not be
 # pushed. Only this file and scripts/autoresearch-frank-archon.sh are
 # committed; the candidate env rides over ssh each run.
 set -uo pipefail
 DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 REMOTE=dankserver
 REMOTE_REPO=/home/dank/serverAppConfigs
 scp -q "$DIR/scripts/frank-archon.env" \
    "$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
 scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
    "$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
 ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
@@ -1,101 +0,0 @@
 # Dankclaw stack secrets and configuration.
 # Copy to .env and fill real values. Never commit real secrets.
 # SearXNG
 SEARXNG_SECRET=changeme
 SEARXNG_BASE_URL=http://search.franksdank.cloud/
 # Kagi search API key (https://help.kagi.com/kagi/api/). Powers the paid
 # `kagi` engine in shared/searxng/settings.yml; deploy-remote.sh bakes it into
 # the live settings.yml via scripts/searxng-render-settings.py. The quadlet
 # does NOT pass it into the container environment.
 KAGI_API_KEY=changeme
 # Gluetun VPN (see https://github.com/qdm12/gluetun-wiki)
 # Single-peer WireGuard model. Fill from your provider's WireGuard config.
 VPN_SERVICE_PROVIDER=protonvpn
 VPN_TYPE=wireguard
 WIREGUARD_PRIVATE_KEY=changeme
 WIREGUARD_ADDRESSES=0.0.0.0/32
 WIREGUARD_PUBLIC_KEY=changeme
 WIREGUARD_ENDPOINT_IP=0.0.0.0
 WIREGUARD_ENDPOINT_PORT=51820
 WIREGUARD_ALLOWED_IPS=0.0.0.0/0,::/0
 WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=25s
 # Gitea database
 GITEA_DB_PASSWORD=changeme
 # Grafana OAuth2 client secret (authentik OAuth2Provider "grafana", pk 9).
 # Created via ak shell on authentik-server; the live .env must hold the
 # provider's generated client_secret verbatim. Rotation = rotate the provider
 # secret in authentik, restart grafana after updating the live .env value.
 GRAFANA_OAUTH_CLIENT_SECRET=changeme
 # Caddy (the homelab's single reverse proxy, on this host)
 # bcrypt hash (caddy hash-password). deploy.sh renders it into
 # /home/dank/Containers/caddy/Caddyfile, which the container mounts.
 # CADDY_BASICAUTH_HASH: the shared family password, now used only by the
 #   pihole vhost (grafana moved behind the Authentik guard chain 2026-09-01).
 #   Username: dankuser.
 CADDY_BASICAUTH_HASH=changeme
 # (The homepage's per-user hashes were retired: the homepage and its /admin/
 #  page are now behind Authentik forward_auth, not basicauth.)
 # Hermes -> Signal (signal-cli daemon on the host, port 8080)
 # E.164 number of the linked Signal account (your primary number, for Note-to-Self).
 SIGNAL_ACCOUNT=+15551234567
 # Comma-separated E.164 numbers allowed to message the agent.
 SIGNAL_ALLOWED_USERS=+15551234567
 # Hermes web UI (Hermex server) login password. Required: the webui binds to
 # loopback and is reached only through the SSH tunnel, but Hermex still signs
 # in with this password. Set a strong value in the live .env.
 HERMES_WEBUI_PASSWORD=changeme
 # Porkbun API (franksdank.cloud + franksdank.casa zones). Caddy uses these
 # for DNS-01 Let's Encrypt wildcard cert issuance. Porkbun API keys are
 # account-wide; rotate them here rather than in the dashboard.
 # PORKBUN_API_SECRET_KEY is the name the caddy-dns/porkbun module expects.
 PORKBUN_API_KEY=changeme
 PORKBUN_API_SECRET_KEY=changeme
 # Authentik (SSO / OIDC provider)
 # AUTHENTIK_PG_PASS: the PostgreSQL password for authentik-postgres.
 #   Generate: openssl rand -base64 32
 AUTHENTIK_PG_PASS=changeme
 # AUTHENTIK_SECRET_KEY: the server's secret key (64+ chars).
 #   Generate: openssl rand -base64 48
 AUTHENTIK_SECRET_KEY=changeme
 # Marreta (content-cleaning proxy, behind the Authentik guard chain).
 # deploy-remote.sh seeds /home/dank/Containers/marreta/env from these three
 # values (only if that file does not already exist) and the marreta quadlet
 # maps them to the app's APP_KEY / ADMIN_EMAIL / ADMIN_PASSWORD.
 # MARRETA_APP_KEY: Laravel encryption key. Generate:
 #   openssl rand -base64 32
 # Changing it after first boot invalidates encrypted session/cookie data.
 MARRETA_APP_KEY=changeme
 # Admin panel login (marreta.franksdank.cloud/admin). The app creates this
 # account on first boot from these values.
 MARRETA_ADMIN_EMAIL=admin@franksdank.cloud
 MARRETA_ADMIN_PASSWORD=changeme
 # open-notebook (podcast generator; internal-only, no vhost).
 # deploy-remote.sh seeds /home/dank/Containers/open-notebook/env from these
 # values (only if that file does not already exist) and refuses to start the
 # units while any of them is blank: a blank OPEN_NOTEBOOK_PASSWORD means
 # authentication OFF, and a blank encryption key means no provider credential
 # can be saved. The seed writes SURREAL_PASS too — the spelling the `surreal`
 # CLI reads — from SURREAL_PASSWORD, so the app and the database cannot be
 # given different values.
 # OPEN_NOTEBOOK_ENCRYPTION_KEY: encrypts the stored AI provider keys at rest.
 #   Generate: openssl rand -hex 32
 # Changing it after credentials are saved makes them undecryptable.
 OPEN_NOTEBOOK_ENCRYPTION_KEY=changeme
 # UI/API password. Every API route — including the episode-audio download the
 # app serves without a session — is open to the LAN while this is unset.
 OPEN_NOTEBOOK_PASSWORD=changeme
 # SurrealDB root password (the root user NAME comes from
 # scripts/open-notebook.env, so this file holds only the secret).
 #   Generate: openssl rand -base64 32
 SURREAL_PASSWORD=changeme
@@ -1,462 +0,0 @@
 # Homelab Caddyfile — the single Caddy for both hosts.
 #
 # Caddy runs as a container on dankclaw (192.168.0.143, this repo's
 # compose.yaml service 'caddy') and fronts every service on both hosts:
 #   - dankclaw services: by container name (compose network) or by the
 #     published LAN port (quadlet services on isolated networks)
 #   - dankserver services (192.168.0.235): by published LAN port
 #
 # Naming: flat franksdank.cloud scheme (see serverAppConfigs-docs/Infrastructure/LAN-DNS.md).
 # DNS for these names is served by the Pi-hole quadlet on this host
 # (dankclaw/pihole/addn-hosts — every name resolves to 192.168.0.143)
 # and handed to LAN clients by the SURFboard G36 router.
 #
 # Authentication chain (2026-08-29): the guarded vhosts (homepage, SearXNG,
 # Prometheus, RomM, Copyparty) proxy EVERYTHING through the Anubis
 # bot-challenge proxy first, then the whole request is proxied THROUGH
 # Authentik (2026.8 proxy-provider mode: this build serves no forward_auth
 # endpoint; the authentication flow runs inline — unauthenticated clients get
 # the login UI served in place, and once a session exists Authentik proxies
 # the request on to the app via the provider's internal_host). Chain: client →
 # Caddy (TLS, public listener) → Anubis :8923 (verify: DENY known AI scrapers,
 # CHALLENGE everyone else) → Caddy's internal listener http://:8016 (auth
 # decision; Host headers survive every hop) → authenticated traffic is
 # forwarded back to http://:8017 and routed straight to the app.
 # Apps with their own auth (Gitea, OpenWebUI, Jellyfin, *arr, Pi-hole) and the
 # Authentik IdP itself (auth.franksdank.cloud — its login flow must never sit
 # behind a challenge) are exempt by construction: their vhosts never route
 # through Anubis. ComfyUI left this exempt set on 2026-08-31 and Grafana left
 # it on 2026-09-01 (OAuth2 relying-party mode against authentik): both now sit
 # behind the guard chain; Grafana authenticates users itself via the OAuth2
 # code flow, so no identity header is forwarded to it.
 # The pihole vhost keeps its basicauth edge; the hash placeholder below is
 # rendered by deploy.sh from CADDY_BASICAUTH_HASH (host .env, git-ignored) into
 # /home/dank/Containers/caddy/Caddyfile, which the container mounts.
 #
 # TLS: real Let's Encrypt certs. Site addresses carry no http:// prefix,
 # so Caddy serves HTTPS (and auto-redirects HTTP). Certs are issued via
 # dns-01 through the Porkbun module built into the franksdank/caddy-porkbun
 # image (caddy/Containerfile). That needs no inbound ports and works from the
 # LAN, where Pi-hole resolves these names to 192.168.0.143.
 {
 	# Admin stays ON: it binds to 127.0.0.1:2019 inside the container and the
 	# port is not published, so nothing outside the container can reach it.
 	# It powers the config file watcher, so a rendered-config change
 	# hot-reloads without a container restart. 'admin off' once left a broken
 	# basicauth hash sitting on disk unapplied for hours (2026-08-22).
 	email admin@franksdank.cloud
 	# Solve ACME challenges for every site via Porkbun DNS (no port 80/443
 	# reachability needed). Credentials come from the container environment
 	# (compose.yaml, from the host .env — never committed).
 	acme_dns porkbun {
 		api_key {env.PORKBUN_API_KEY}
 		api_secret_key {env.PORKBUN_API_SECRET_KEY}
 	}
 }
 # --- Bare-IP access (health checks, convenience) ---
 :80 {
 	redir https://www.franksdank.cloud/ 308
 }
 # --- Homepage (Anubis → Authentik SSO) ---
 # www holds the homepage; the naked domain redirects onto it at the TLS edge.
 # A redirect needs no challenge, so unauthenticated naked traffic never enters
 # the auth chain. The proxy provider's external_host must match the host that
 # actually carries traffic (https://www.franksdank.cloud) or Authentik matches
 # zero providers and answers its error page instead.
 franksdank.cloud {
 	redir https://www.franksdank.cloud/ 308
 }
 www.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Legacy name: the old homepage name redirects straight onto www ---
 dankclaw.franksdank.cloud {
 	redir https://www.franksdank.cloud/ 308
 }
 # --- Legacy name: the media host has no Caddy of its own anymore ---
 dankserver.franksdank.cloud {
 	redir https://www.franksdank.cloud/ 308
 }
 # --- Authentik (SSO / OIDC provider) ---
 # Exempt by construction: the IdP login flow must be reachable without a
 # challenge, so this vhost never routes through Anubis.
 auth.franksdank.cloud {
 	reverse_proxy authentik-server:9000
 }
 # ═══════════════════════════════════════════════════════════════
 # Services on this host (dankclaw, 192.168.0.143)
 # ═══════════════════════════════════════════════════════════════
 # --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
 search.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
 ai.franksdank.cloud {
 	reverse_proxy 192.168.0.143:3005
 }
 # --- Gitea (quadlet; published 3000) ---
 # Exempt by construction: Gitea has its own login, and git smart-HTTP
 # clients (git push/fetch over HTTPS) cannot solve JS challenges. The
 # policy would need a bypass for /info/refs + /git-*-pack paths if this
 # ever moves behind Anubis.
 gitea.franksdank.cloud {
 	reverse_proxy 192.168.0.143:3000
 }
 # --- Grafana (guarded: Anubis → Authentik outpost gate; login inside Grafana) ---
 # Auth model = OAuth2 relying party against authentik (OAuth2Provider "grafana",
 # confidential client). The guard chain gates unauthenticated traffic; the
 # outpost carries no provider token in this deployment, so identity headers
 # never reach Grafana — RP mode needs none and works regardless of topology.
 # The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
 grafana.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Prometheus ---
 prometheus.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
 loki.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
 seerr.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
 # Content-cleaning proxy. Any authentik-authenticated user may use it (no
 # group gate). Egress rides the searxng gluetun tunnel; reached via the
 # host-published port 8082 (gluetun netns), not by container name.
 marreta.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Open Notebook (guarded: Anubis → Authentik; notebook-proxy provider, internal_host caddy:8017) ---
 # The podcast generator. ONE upstream carries the whole app: the image's own
 # Next.js layer rewrites /api/* to the FastAPI beside it (INTERNAL_API_URL,
 # default http://localhost:5055 inside the container), so a browser on this
 # vhost never needs the API port. That is why scripts/open-notebook.env sets
 # API_URL to exactly this origin — left unset, the app auto-detects
 # `<host-you-opened>:5055`, and 5055 on this host belongs to seerr — and why
 # the published API port sits on loopback. App-level login
 # (OPEN_NOTEBOOK_PASSWORD) stays on behind the chain: two owners, edge identity
 # and the app's own session.
 # Dialled by published LAN port, not container name: open-notebook is a
 # rootless quadlet and this Caddy lives in root's storage, so root's DNS does
 # not see the container (the same reason marreta and SearXNG are dialled by IP).
 notebook.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Pi-hole (quadlet on slirp4netns; published 8083) ---
 pihole.franksdank.cloud {
 	basicauth {
 		dankuser REPLACE_WITH_HASH
 	}
 	reverse_proxy 192.168.0.143:8083
 }
 # --- Hermes WebUI (Hermex webui; host-netns container, published 8787) ------
 # Exempt by construction: the webui authenticates itself (HERMES_WEBUI_PASSWORD),
 # so no Anubis challenge and no Authentik proxy provider is required — nothing
 # here enters the http://:8016 auth chain. Internal-only by DNS absence: there is
 # no public Porkbun A record for this name, so only LAN clients (Pi-hole
 # addn-hosts → 192.168.0.143) can resolve it at all; DNS-01 issuance via the
 # Porkbun module needs no public record. Upstream is `host.containers.internal`
 # — netavark's host address, 169.254.1.2 — NOT the host LAN IP: hermes-webui is
 # `Network=host`, so 8787 is a host-netns bind and not a published port, and
 # netavark DNATs only published ports (192.168.0.143:9119, the dashboard below,
 # answers from this bridge; 192.168.0.143:8787 does not). Measured live
 # 2026-09-16 inside the Caddy container: 192.168.0.143:8787 and 10.89.1.1:8787
 # both REFUSED, host.containers.internal:8787 → 200.
 hermes.franksdank.cloud {
 	reverse_proxy host.containers.internal:8787
 }
 # --- Hermes dashboard (admin UI; published 9119 on dankclaw_default) --------
 # Exempt by construction: the app enforces its own complete auth — verified live
 # from Caddy's own network, an unauthenticated GET / returns 302 → /login?next=%2F
 # (uvicorn), and basic_auth.username + a scrypt password_hash are set in the
 # dashboard's /data/config.yaml. No outpost provider needed.
 hermesadmin.franksdank.cloud {
 	reverse_proxy 192.168.0.143:9119
 }
 # ═══════════════════════════════════════════════════════════════
 # Services on dankserver (192.168.0.235) — published LAN ports
 # ═══════════════════════════════════════════════════════════════
 # --- Plex (existing, kept) ---
 # http:// on purpose: franksdank.casa is not hosted in the same Porkbun
 # account as franksdank.cloud, so the DNS-01 provider cannot issue a
 # Let's Encrypt cert for it. It stays plain-HTTP legacy; the canonical
 # name is plex.franksdank.cloud (HTTPS, below).
 http://plex.franksdank.casa {
 	reverse_proxy 192.168.0.235:32400 {
 		transport http {
 			tls_insecure_skip_verify
 		}
 	}
 }
 # PMS requires TLS from every non-loopback client: plain-HTTP off-loopback
 # is read and silently closed (diagnosed 2026-08-26), which surfaced as an
 # empty-body 502 here. Dial the upstream over TLS; PMS's self-signed cert is
 # skipped on this pinned-LAN hop only (client-facing LE TLS is untouched).
 # The web UI lives at /web/; / returns the XML MediaContainer index.
 plex.franksdank.cloud {
 	reverse_proxy 192.168.0.235:32400 {
 		transport http {
 			tls_insecure_skip_verify
 		}
 	}
 }
 # --- Calibre-Web ---
 books.franksdank.cloud {
 	reverse_proxy 192.168.0.235:8083
 }
 # --- RomM ---
 romm.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Copyparty ---
 files.franksdank.cloud {
 	reverse_proxy anubis:8923 {
 		header_up X-Real-Ip {remote_host}
 	}
 }
 # --- Jellyfin ---
 jelly.franksdank.cloud {
 	reverse_proxy 192.168.0.235:8096
 }
 # --- *arr Stack (exempt by construction: own auth) ---
 prowlarr.franksdank.cloud {
 	reverse_proxy 192.168.0.112:9696
 }
 radarr.franksdank.cloud {
 	reverse_proxy 192.168.0.112:7878
 }
 # --- Sonarr: internal-only by construction (retired 2026-09-04) -----------
 # No vhost and no public DNS record for sonarr.franksdank.cloud: the TV-library
 # admin UI is reachable only on the LAN at 192.168.0.112:8989 (users request TV
 # through Seerr, which sits behind the guard chain; admins reach Sonarr's own
 # UI directly). A brief unguarded public vhost lived here and served the
 # zero-auth UI to the internet via manual DNS — retired.
 lidarr.franksdank.cloud {
 	reverse_proxy 192.168.0.112:8686
 }
 readarr.franksdank.cloud {
 	reverse_proxy 192.168.0.112:8787
 }
 qbittorrent.franksdank.cloud {
 	reverse_proxy 192.168.0.112:8080
 }
 # --- Internal listener: auth decision layer ---------------------------------
 # Anubis dials http://caddy:8016 after verifying a client (TARGET in
 # dankclaw/anubis/quadlet/anubis.container). Host headers survive the hop.
 # Every request here is proxied whole-request to Authentik with Host
 # preserved. This build's embedded outpost routes ONLY on the ORIGINAL Host
 # against each proxy provider's external_host — meta headers are ignored and
 # cross-host absolute next= redirects are rejected by the flow's open-redirect
 # guard ("Request has been denied: Invalid next URL"), verified live.
 # Unauthenticated clients get the login flow served in place; authenticated
 # sessions are forwarded by the provider's internal_host back to
 # http://:8017 and routed straight to the app. Every host that may enter this
 # listener MUST have a proxy provider registered with the embedded outpost —
 # the Anubis guard set, the Caddy TLS edge, and the provider list stay in
 # lockstep. Exempt vhosts never enter this block.
 http://:8016 {
 	reverse_proxy authentik-server:9000 {
 		header_up X-Forwarded-Proto https
 	}
 }
 # --- Post-auth listener: where Authentik forwards authenticated traffic ----
 # The proxy provider's internal_host is http://caddy:8017, so every request
 # that passes authentication lands here (Host header preserved) and routes
 # straight to its app. This is the end of the chain — nothing dials back
 # into Authentik from here, so the chain cannot loop.
 http://:8017 {
 	@home host www.franksdank.cloud
 	handle @home {
 		root * /srv/homepage
 		# Deploys overwrite /srv/homepage in place; without revalidation a
 		# browser keeps serving the pre-deploy CSS/JS/HTML. no-cache forces an
 		# ETag round-trip on every load (a cheap 304 when unchanged).
 		header Cache-Control "no-cache"
 		# RBAC (2026-09-01): /admin requires membership in the homepage-admins
 		# authentik group. In proxy-provider mode the outpost injects
 		# X-Authentik-Groups on its upstream request to this listener and strips
 		# any client-supplied X-Authentik-* headers first, so browsers cannot
 		# spoof it. Regex is a substring match, insensitive to the group-list
 		# separator.
 		handle /admin* {
 			@adminok header_regexp X-Authentik-Groups homepage-admins
 			handle @adminok {
 				try_files /admin.html
 				file_server
 			}
 			respond "Forbidden: homepage admins only" 403
 		}
 		# User docs page (static). Anchors (#models etc.) are client-side,
 		# so both /docs and /docs/ map onto the single docs.html.
 		handle /docs {
 			try_files /docs.html
 			file_server
 		}
 		handle /docs/ {
 			try_files /docs.html
 			file_server
 		}
 		# /whoami echoes the outpost-injected identity headers so the page
 		# can gate the admin button client-side (the /admin* 403 above stays
 		# the real gate; this only decides what is shown). The outpost strips
 		# client-supplied X-Authentik-* before injecting, so this is spoof-proof.
 		handle /whoami {
 			templates
 			header Content-Type application/json
 			respond "{\"user\":\"{header.X-Authentik-Username}\",\"groups\":\"{header.X-Authentik-Groups}\"}"
 		}
 		@data path /services.json /status.json /app.js /style.css /deployed.js /img/* /sfx/*
 		handle @data {
 			file_server
 		}
 		handle {
 			try_files /index.html
 			file_server
 		}
 	}
 	@search host search.franksdank.cloud
 	handle @search {
 		reverse_proxy 192.168.0.143:8081
 	}
 	@loki host loki.franksdank.cloud
 	handle @loki {
 		reverse_proxy loki:3100
 	}
 	@seerr host seerr.franksdank.cloud
 	handle @seerr {
 		reverse_proxy seerr:5055
 	}
 	@grafana host grafana.franksdank.cloud
 	handle @grafana {
 		reverse_proxy grafana:3000
 	}
 	@prometheus host prometheus.franksdank.cloud
 	handle @prometheus {
 		reverse_proxy prometheus:9090
 	}
 	@romm host romm.franksdank.cloud
 	handle @romm {
 		reverse_proxy 192.168.0.235:8085
 	}
 	@files host files.franksdank.cloud
 	handle @files {
 		reverse_proxy 192.168.0.235:3923
 	}
 	@prowlarr host prowlarr.franksdank.cloud
 	handle @prowlarr {
 		reverse_proxy 192.168.0.112:9696
 	}
 	@radarr host radarr.franksdank.cloud
 	handle @radarr {
 		reverse_proxy 192.168.0.112:7878
 	}
 	@lidarr host lidarr.franksdank.cloud
 	handle @lidarr {
 		reverse_proxy 192.168.0.112:8686
 	}
 	@readarr host readarr.franksdank.cloud
 	handle @readarr {
 		reverse_proxy 192.168.0.112:8787
 	}
 	@qbittorrent host qbittorrent.franksdank.cloud
 	handle @qbittorrent {
 		reverse_proxy 192.168.0.112:8080
 	}
 	@marreta host marreta.franksdank.cloud
 	handle @marreta {
 		reverse_proxy 192.168.0.143:8082
 	}
 	@notebook host notebook.franksdank.cloud
 	handle @notebook {
 		# Bare proxy, like ai.franksdank.cloud: the app's SSE answers
 		# (/api/search/ask, chat sessions) stream for minutes through this
 		# same listener with no flush or timeout tuning, so nothing here
 		# needs it — a read/write timeout on the route would be the thing
 		# that cuts them.
 		reverse_proxy 192.168.0.143:8502
 	}
 }
 # --- Optional: Jackett ---
 # jackett.franksdank.cloud {
 # 	reverse_proxy 192.168.0.235:9117
 # }
 # --- Optional: Whisparr ---
 # whisparr.franksdank.cloud {
 # 	reverse_proxy 192.168.0.235:6969
 # }
@@ -1,40 +0,0 @@
 # ComfyUI on the gfx1151 (AMD Strix Halo / Radeon 8060S) ROCm torch build.
 #
 # Base image: AMD's gfx1151-tagged vLLM image. It is the only verified build
 # where torch.cuda.is_available() is True on this APU (device capability
 # (11,5) = gfx1151). It already ships the matching ROCm torch family
 # (torch 2.9.1+rocm, torchvision 0.24.0+rocm, torchaudio 2.9.0+rocm), so we
 # must NOT let pip reinstall them: ComfyUI's requirements.txt lists an unpinned
 # `torch`, which would otherwise clobber the gfx1151 kernels with a default build.
 FROM docker.io/rocm/vllm:rocm7.12.0_gfx1151_ubuntu24.04_py3.12_pytorch_2.9.1_vllm_0.16.0
 # Bake the ComfyUI code into the image (rebuild the image to update the code).
 RUN git clone --depth 1 https://github.com/comfyanonymous/ComfyUI /opt/ComfyUI
 WORKDIR /opt/ComfyUI
 # Pre-create the user-data dirs. The host mounts its collections over them
 # (see the comfyui service in compose.yaml).
 RUN mkdir -p /opt/ComfyUI/models /opt/ComfyUI/input /opt/ComfyUI/output /opt/ComfyUI/custom_nodes
 # Install ComfyUI's dependencies, but skip the torch family (already present
 # above as the gfx1151 ROCm builds). torchsde and the rest are kept.
 RUN grep -viE '^(torch|torchvision|torchaudio)([^a-z]|$)' requirements.txt > /tmp/req-notorch.txt \
    && pip install --no-cache-dir -r /tmp/req-notorch.txt
 # The city96/ComfyUI-GGUF custom node (loaded from the host's custom_nodes
 # volume) needs the gguf library to decode GGUF-quantized diffusion models.
 # sentencepiece and protobuf are its optional tokenizer deps. Baking them into
 # the image keeps the dependency across container recreations — the node itself
 # lives in the host volume, which persists, but the Python env does not.
 RUN pip install --no-cache-dir "gguf>=0.13.0" sentencepiece protobuf
 # Bake the qwen_image21 patch script into the image. It is run by the
 # startup wrapper before ComfyUI starts, so the patch survives container
 COPY dankclaw/apply_qwen21_patch.py /opt/ComfyUI/apply_qwen21_patch.py
 COPY dankclaw/comfyui-start.sh /opt/ComfyUI/comfyui-start.sh
 RUN chmod +x /opt/ComfyUI/comfyui-start.sh
 CMD ["/opt/ComfyUI/comfyui-start.sh"]
 EXPOSE 8188
 # v2: VAE 4D-to-5D latent expansion for video VAEs (qwen_image21)
 # v3: fix in_channels to use shape[0] (latent input dim) not shape[1] (hidden dim)
@@ -1,18 +0,0 @@
 # Alloy + a static busybox.
 #
 # grafana/alloy has a shell but no wget/curl, so the compose healthcheck
 # cannot run anything inside the container. This image drops a static
 # busybox into it for the healthcheck command only. All Alloy behavior is
 # unchanged.
 #
 # Build on dankclaw:
 #   podman build -t franksdank/alloy:latest -f dankclaw/alloy/Containerfile .
 FROM docker.io/library/alpine:latest AS tools
 # busybox-static is statically linked: the target images have no dynamic
 # loader (loki is distroless), so the alpine default /bin/busybox (musl
 # dynamic) would fail with "no such file or directory".
 RUN apk add --no-cache busybox-static
 FROM docker.io/grafana/alloy:latest
 COPY --from=tools /bin/busybox.static /usr/local/bin/busybox
@@ -1,25 +0,0 @@
 {
  "bots": [
    {
      "name": "deny AI scrapers",
      "user_agent_regex": "(?i)^(GPTBot|ChatGPT-User|OAI-SearchBot|CCBot|anthropic-ai|ClaudeBot|Claude-Web|PerplexityBot|PerplexityUser|Google-Extended|Applebot-Extended|Bytespider|Amazonbot|meta-externalagent|DuckAssistBot|cohere-ai|YouBot|KimiBot)",
      "action": "DENY"
    },
    {
      "name": "serve anubis own endpoints",
      "path_regex": "^/\\.within\\.website",
      "action": "ALLOW"
    },
    {
      "name": "serve authentik outpost transport",
      "path_regex": "^/outpost\\.goauthentik\\.io/",
      "action": "ALLOW"
    },
    {
      "name": "challenge everything else",
      "path_regex": ".",
      "action": "CHALLENGE"
    }
  ],
  "dnsbl": false
 }
@@ -1,70 +0,0 @@
 # Quadlet unit: Anubis bot-challenge proxy (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/anubis.container
 # Unit name: anubis.service (container name: anubis)
 #
 # Two-tier chain: the public Caddy listeners for the guarded vhosts proxy
 # everything to this container (port 8923, internal only — no published port,
 # so metrics on :9090 stay reachable only inside dankclaw_default); Anubis
 # verifies the client (challenge/DENY per policy) and forwards accepted
 # traffic back to Caddy's internal listener (TARGET below), where the normal
 # per-vhost routing — including Authentik forward_auth — resumes. Host headers
 # survive both hops, so vhost matching on the internal listener still works.
 #
 # Image pin: the image is pulled from ghcr.io/techarohq/anubis (source of
 # truth), but the unit runs it under a localhost/-aliased tag: on this host's
 # podman 5.7 quadlet path, registry-prefixed refs fail to resolve from local
 # storage at container create time ("image not known") even when present; the
 # localhost alias resolves (verified live 2026-08-28). v1.27.0 pinned 2026-09-02: --help
 # verified on the image itself (cookie-domain, ed25519-private-key-hex,
 # cookie-expiration-time default 168h, difficulty default 4); JSON policy
 # schema unchanged; X-Real-Ip is now REQUIRED (missing header => HTTP 500) —
 # the Caddy chain always sets it (bats seam enforces the header_up line).
 # Upgrading = pull new tag, re-tag the localhost alias, restart the unit.
 #
 # No secrets in this repo. COOKIE_DOMAIN makes the challenge cookie valid for
 # all *.franksdank.cloud vhosts: solve the proof-of-work once (e.g. on www)
 # and every guarded vhost honors it until expiry (default 168h, aligned with
 # authentik's 7-day session) or re-login. ED25519_PRIVATE_KEY_HEX pins the
 # JWT signing key (host file /home/dank/Containers/anubis/env, mode 0600,
 # gitea-style ${VAR} expansion via the [Service] EnvironmentFile): without
 # it every restart invalidates everyone's solved challenges. See
 # serverAppConfigs-docs/Infrastructure/Anubis.md.
 [Unit]
 Description=Anubis bot-challenge proxy container
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=anubis
 Image=localhost/techarohq/anubis:v1.27.0
 Pull=never
 Network=dankclaw_default
 # flagenv maps env names to flag names (TARGET -> --target, POLICY_FNAME ->
 # --policy-fname). TARGET points back at Caddy's internal listener; the
 # container DNS name resolves on dankclaw_default.
 Environment=TARGET=http://caddy:8016
 Environment=POLICY_FNAME=/policy/policy.json
 Environment=COOKIE_DOMAIN=franksdank.cloud
 EnvironmentFile=/home/dank/Containers/anubis/env
 Environment=ED25519_PRIVATE_KEY_HEX=${ED25519_PRIVATE_KEY_HEX}
 Volume=/home/dank/serverAppConfigs/dankclaw/anubis/policy.json:/policy/policy.json:ro
 # The image ships no HEALTHCHECK (distroless ko-build: entrypoint is
 # /ko-app/anubis, no shell in the image), so the probe is declared here.
 # JSON exec form on purpose: this quadlet generator joins HealthCmd into a
 # single \x20-escaped argv element and podman then runs it as shell-form —
 # which needs a shell the distroless image does not have. A JSON array is
 # passed through verbatim and executes /ko-app/anubis directly.
 HealthCmd=["/ko-app/anubis","--healthcheck"]
 HealthInterval=15s
 HealthTimeout=5s
 HealthRetries=3
 [Service]
 EnvironmentFile=/home/dank/Containers/anubis/env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,52 +0,0 @@
 import os
 # Apply qwen_image21 detection patch
 f = "/opt/ComfyUI/comfy/model_detection.py"
 with open(f) as fh:
    code = fh.read()
 if "qwen_image21" not in code:
    qwen21_check = (
        "    if '{}txt_in.text_norm.weight'.format(key_prefix) in state_dict_keys:  # Qwen Image 2.1\n"
        "        dit_config = {}\n"
        '        dit_config["image_model"] = "qwen_image21"\n'
        "        dit_config[\"in_channels\"] = state_dict['{}img_in.weight'.format(key_prefix)].shape[0]\n"
        "        dit_config[\"num_layers\"] = count_blocks(state_dict_keys, '{}transformer_blocks.'.format(key_prefix) + '{}.')\n"
        "        return dit_config\n"
    )
    anchor = "    if '{}txt_norm.weight'.format(key_prefix) in state_dict_keys:  # Qwen Image"
    code = code.replace(anchor, qwen21_check + "\n" + anchor)
    with open(f, "w") as fh:
        fh.write(code)
    print("Applied qwen_image21 detection patch")
 # Add qwen_image21 to GGUF node arch list
 loader = "/opt/ComfyUI/custom_nodes/ComfyUI-GGUF/loader.py"
 if os.path.exists(loader):
    with open(loader) as fh:
        lcode = fh.read()
    if '"qwen_image21"' not in lcode:
        old = '"lumina2", "qwen_image"}'
        new = '"lumina2", "qwen_image", "qwen_image21"}'
        if old in lcode:
            lcode = lcode.replace(old, new)
            with open(loader, "w") as fh:
                fh.write(lcode)
            print("Applied qwen_image21 arch list patch")
 # Patch VAE decode to expand 4D latents to 5D for video VAEs (latent_dim=3)
 sd = "/opt/ComfyUI/comfy/sd.py"
 with open(sd) as fh:
    sdc = fh.read()
 if "self.latent_dim == 3 and samples_in.ndim == 4" not in sdc:
    anchor = "        if self.latent_dim == 2 and samples_in.ndim == 5:\n            samples_in = samples_in[:, :, 0]"
    replacement = (
        "        if self.latent_dim == 2 and samples_in.ndim == 5:\n"
        "            samples_in = samples_in[:, :, 0]\n"
        "        elif self.latent_dim == 3 and samples_in.ndim == 4:\n"
        "            samples_in = samples_in.unsqueeze(2)"
    )
    if anchor in sdc:
        sdc = sdc.replace(anchor, replacement)
        with open(sd, "w") as fh:
            fh.write(sdc)
        print("Applied VAE 4D-to-5D latent expansion patch")
@@ -1,40 +0,0 @@
 # Quadlet unit: Authentik LDAP outpost (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/authentik-outpost-ldap.container
 # Unit name: authentik-outpost-ldap.service (podman 5.x drops the container- prefix)
 #
 # Serves the authentik LDAP provider ("Jellyfin LDAP", Base DN
 # dc=franksdank,dc=cloud) so dankserver's Jellyfin can authenticate its users
 # via the LDAP-Auth plugin. The type-specific ghcr.io/goauthentik/ldap image
 # listens on 3389 (ldap) / 6636 (ldaps) inside the container; published to
 # host 389/636 for cross-host binds from dankserver. The outpost token lives
 # in the host .env (AUTHENTIK_LDAP_OUTPOST_TOKEN), never in this repo. The
 # outpost dials core over websocket at AUTHENTIK_HOST; auth. is guard-chain
 # exempt (IdP).
 [Unit]
 Description=Authentik LDAP outpost container
 After=network.target authentik-server.service
 Wants=authentik-server.service
 [Container]
 ContainerName=authentik-outpost-ldap
 Network=dankclaw_default
 # Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
 # the tag's nested manifest list, so pin the single-platform digest).
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
 Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
 Environment=AUTHENTIK_INSECURE=false
 Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}
 PublishPort=389:3389
 PublishPort=636:6636
 User=0:0
 [Service]
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,44 +0,0 @@
 # Quadlet unit: Authentik outpost (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/authentik-outpost.container
 # Unit name: authentik-outpost.service (podman 5.x drops the container- prefix)
 #
 # In authentik 2026.8.x the proxy-provider URL space (/outpost.goauthentik.io/*)
 # is served by the Rust worker binary, not the server — verified live: with a
 # tenant-matching Host header, /if/flow/... answers 200 on the server while
 # /auth/caddy still 404s. This container runs the same image as the server and
 # worker units; Exec=worker selects the Rust worker role, and the
 # AUTHENTIK_HOST/AUTHENTIK_TOKEN env flips it into outpost mode: it dials back
 # to core over websocket (ws/outpost/<uuid>/) and serves forward-auth for Caddy
 # on its own 9000. No published port — Caddy reaches it by container DNS on
 # dankclaw_default; the token lives in the host .env, never in this repo.
 [Unit]
 Description=Authentik outpost container
 After=network.target authentik-postgres.service authentik-server.service
 Wants=authentik-server.service
 [Container]
 ContainerName=authentik-outpost
 Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
 Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
 Environment=AUTHENTIK_POSTGRESQL__USER=authentik
 Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
 Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
 Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
 Environment=AUTHENTIK_INSECURE=false
 Environment=AUTHENTIK_TOKEN=${AUTHENTIK_OUTPOST_TOKEN}
 Volume=/home/dank/Containers/authentik/data:/data
 User=0:0
 [Service]
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,42 +0,0 @@
 # Quadlet unit: Authentik PostgreSQL 16 (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/authentik-postgres.container
 # Unit name: authentik-postgres.service (podman 5.x drops the container- prefix)
 #
 # Replaces the compose service retired when this stack went boot-persistent
 # (rootless Podman does not auto-start bare containers at boot). Secrets stay
 # out of this file: systemd expands ${AUTHENTIK_PG_PASS} from the git-ignored
 # host .env at start time (same pattern as gitea-postgres).
 [Unit]
 Description=Authentik PostgreSQL container
 After=network.target
 [Container]
 ContainerName=authentik-postgres
 Image=docker.io/library/postgres:16-alpine
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=POSTGRES_DB=authentik
 Environment=POSTGRES_USER=authentik
 Environment=POSTGRES_PASSWORD=${AUTHENTIK_PG_PASS}
 Volume=/home/dank/Containers/authentik/postgres:/var/lib/postgresql/data
 # Unquoted argv: this quadlet generator space-splits HealthCmd into the
 # healthcheck's argv array, so a quoted probe would arrive mangled (the old
 # `/bin/sh -c "pg_isready..."` form arrived as /bin/sh + broken args and
 # failed with exit 1 forever — the health status was a false negative while
 # the DB itself accepted connections). Bare argv works under both exec forms.
 HealthCmd=pg_isready -d authentik -U authentik
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=5
 HealthStartPeriod=20s
 [Service]
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 TimeoutStartSec=900
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,45 +0,0 @@
 # Quadlet unit: Authentik core + embedded Caddy outpost (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/authentik-server.container
 # Unit name: authentik-server.service (podman 5.x drops the container- prefix)
 #
 # The embedded outpost serves /outpost.goauthentik.io/* including the
 # /auth/caddy forward_auth endpoint the Caddyfile chains to — no outpost token
 # exists for this deployment by design. Resolved by name (A2A DNS) from every
 # container on dankclaw_default, including caddy. The 9000 publish is
 # loopback-only on purpose: LAN clients reach Authentik through Caddy, never
 # around it; the loopback publish exists so host-side tooling can probe
 # /-/health/. Runs as the image default user (same userns mapping the compose
 # era used, so the existing /data ownership carries over unchanged).
 [Unit]
 Description=Authentik server container
 After=network.target authentik-postgres.service
 Wants=authentik-postgres.service
 [Container]
 ContainerName=authentik-server
 Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=server
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
 Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
 Environment=AUTHENTIK_POSTGRESQL__USER=authentik
 Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
 Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
 PublishPort=127.0.0.1:9000:9000
 # Probe must contain no quotes: this host's quadlet v1 generator space-splits
 # HealthCmd into the healthcheck argv array, and quoted probes arrive mangled
 # (verified live 2026-08-28). `python /manage.py check --database default`
 # exits nonzero iff Django (incl. the DB connection) cannot serve.
 HealthCmd=python /manage.py check --database default
 [Service]
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 TimeoutStartSec=900
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,37 +0,0 @@
 # Quadlet unit: Authentik worker — background tasks, email, webhook/log outbox
 # (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/authentik-worker.container
 # Unit name: authentik-worker.service (podman 5.x drops the container- prefix)
 #
 # Same image as the server; the Exec= command selects the worker role. Runs as
 # user 0:0 (rootless: maps to the invoking host user, matching the compose-era
 # semantics). No published ports — the worker talks to PostgreSQL in-network
 # and has no listener of its own.
 [Unit]
 Description=Authentik worker container
 After=network.target authentik-postgres.service authentik-server.service
 Wants=authentik-server.service
 [Container]
 ContainerName=authentik-worker
 Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
 Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
 Environment=AUTHENTIK_POSTGRESQL__USER=authentik
 Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
 Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
 Volume=/home/dank/Containers/authentik/data:/data
 User=0:0
 [Service]
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,22 +0,0 @@
 # Caddy with the Porkbun DNS provider (github.com/caddy-dns/porkbun).
 #
 # The porkbun module lets Caddy solve Let's Encrypt DNS-01 challenges
 # directly through the Porkbun API, so the wildcard cert for
 # *.franksdank.cloud / *.franksdank.casa can be issued and renewed without
 # any port 80/443 reachability from the internet (see the Caddyfile in
 # ../Caddyfile and serverAppConfigs-docs/Infrastructure/LAN-DNS.md).
 #
 # Build on dankclaw (podman rootless, same as every other container here):
 #   podman build -t franksdank/caddy-porkbun:latest -f dankclaw/caddy/Containerfile .
 #
 # The first build runs an xcaddy Go compilation and takes a few minutes.
 # Rebuild only when Caddy or the porkbun module must be updated.
 FROM docker.io/library/caddy:2-builder AS builder
 RUN xcaddy build \
    --with github.com/caddy-dns/porkbun \
    --output /usr/bin/caddy-porkbun
 FROM docker.io/library/caddy:2
 COPY --from=builder /usr/bin/caddy-porkbun /usr/bin/caddy
@@ -1,7 +0,0 @@
 # Caddy environment (placeholders only). The deploy writes the live file at
 # /home/dank/Containers/caddy/env from the checkout's .env; the quadlet reads
 # it via EnvironmentFile. Consumed by the Caddyfile's acme_dns block
 # (DNS-01 wildcard cert issuance). PORKBUN_API_SECRET_KEY is the name the
 # caddy-dns/porkbun module expects.
 PORKBUN_API_KEY=changeme
 PORKBUN_API_SECRET_KEY=changeme
@@ -1,54 +0,0 @@
 # Quadlet unit: canonical Caddy reverse proxy + ACME (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/caddy.container
 # Unit name: caddy.service (container name: caddy)
 #
 # The homelab's single Caddy: it fronts every service on both hosts and issues
 # Let's Encrypt wildcard certs via DNS-01 (Porkbun). It left compose.yaml on
 # 2026-08-26 after a reboot exposed that the bare compose container never came
 # back: rootless Podman has no boot-time auto-start for bare containers, and
 # `restart: unless-stopped` does not survive the explicit stop that shutdown
 # performs. See AGENTS.md "Boot Persistence & Self-Healing" and
 # serverAppConfigs-docs/Infrastructure/Caddy.md.
 #
 # It joins the existing compose network (dankclaw_default) so the Caddyfile's
 # container-name upstreams (authentik-server, grafana, ...) still resolve; if
 # no compose service has created that network yet at boot, quadlet creates it
 # first and compose reuses it by name later. The image is built locally from
 # dankclaw/caddy/Containerfile and exists only on this host, so Pull=never: a
 # boot must not stall on a registry that does not have it. Porkbun credentials
 # come from /home/dank/Containers/caddy/env (deploy writes it from the
 # checkout's .env; secrets never enter the repo).
 [Unit]
 Description=Caddy reverse proxy + ACME (Porkbun DNS-01) container
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=caddy
 Image=franksdank/caddy-porkbun:latest
 Pull=never
 Network=dankclaw_default
 PublishPort=0.0.0.0:80:80/tcp
 PublishPort=0.0.0.0:443:443/tcp
 EnvironmentFile=/home/dank/Containers/caddy/env
 Volume=/home/dank/Containers/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
 Volume=/home/dank/Containers/caddy/data:/data
 Volume=/home/dank/Containers/caddy/config:/config
 # The homepage: the host's /srv/homepage (deploy-managed, not the repo's
 # shared/homepage).
 Volume=/srv/homepage:/srv/homepage:ro
 # The bare-IP catch-all answers 302 (curl -f passes on <400); this proves
 # Caddy itself is serving without touching any upstream.
 HealthCmd=curl -sf -o /dev/null http://localhost:80/
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,4 +0,0 @@
 #!/bin/bash
 # Apply qwen_image21 patches before starting ComfyUI
 python3 /opt/ComfyUI/apply_qwen21_patch.py 2>/dev/null
 exec python main.py --listen 0.0.0.0 --port 8188
@@ -1,21 +0,0 @@
 #!/usr/bin/env python3
 """ComfyUI container healthcheck: the UI must answer on :8188.
 Runs INSIDE the container, where the UI listens (the unit publishes that port on
 the host loopback for the homepage status card). ComfyUI only answers once
 torch, the custom nodes and the graph are loaded, so a 200 here means the
 service is usable rather than merely a live process — the distinction that
 matters, because the failure this replaced was a container that stayed "up"
 while dying on every start.
 HealthCmd in the quadlet splits on spaces into an argv array (podman 5.x v1
 generator), which is why this is a file and not an inline `python3 -c`.
 """
 import sys
 import urllib.request
 try:
    with urllib.request.urlopen("http://127.0.0.1:8188/", timeout=10) as resp:
        sys.exit(0 if resp.status == 200 else 1)
 except Exception:
    sys.exit(1)
@@ -1,70 +0,0 @@
 # Quadlet unit: ComfyUI (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/comfyui.container
 # Unit name: comfyui.service (container name: comfyui)
 #
 # WHY THIS FILE EXISTS. ComfyUI ran from `dankclaw/compose.yaml` through a
 # hand-written `~/.config/systemd/user/comfyui.service` that called
 # `podman compose up -d`. Its container was created 2026-09-15 14:12, BEFORE the
 # GPU declarations were added to the compose service, and `restart:
 # unless-stopped` then kept restarting that stale container — so it started
 # without /dev/kfd, without /dev/dri and without keep-groups, died on
 # `RuntimeError: No HIP GPUs are available`, and had restarted 9,146 times when
 # this was found. A recreate via compose would have fixed the container; a unit
 # fixes the class: one owner per container name, repo-tracked, boot-persistent
 # through the generator.
 #
 # GPU ACCESS. /dev/kfd + /dev/dri plus keep-groups. A literal `render` group
 # entry cannot work under rootless Podman (the group name does not resolve
 # inside the user namespace), and the companion udev rule
 # (dankclaw/udev/99-comfyui-gpu-access.rules) keeps those nodes MODE 0666 so a
 # user-unit spawn — which carries no supplementary groups — can still open them.
 # Both halves are asserted by tests/comfyui_rocm.bats.
 #
 # EXPOSURE. LAN-reachable, no auth. ComfyUI has no authentication and executes
 # arbitrary node graphs. It is internal-only (no public vhost, no guard chain);
 # it is reached on the LAN via the Pi-hole DNS record
 # (comfyui.franksdank.cloud -> 192.168.0.143, in dankclaw/pihole/addn-hosts),
 # which resolves to this host's IP — so the bind is widened from loopback to
 # 0.0.0.0:8188. The homepage admin card links to that URL and probes
 # 127.0.0.1:8188.
 #
 # DATA. Models, input, output and custom_nodes live under
 # /home/dank/comfyui-data (host state, not in the repo).
 [Unit]
 Description=ComfyUI (gfx1151 ROCm image generation)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=comfyui
 Image=localhost/comfyui-rocm:gfx1151
 Pull=never
 Network=dankclaw_default
 PublishPort=0.0.0.0:8188:8188
 AddDevice=/dev/kfd
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 Volume=/home/dank/comfyui-data/models:/opt/ComfyUI/models
 Volume=/home/dank/comfyui-data/input:/opt/ComfyUI/input
 Volume=/home/dank/comfyui-data/output:/opt/ComfyUI/output
 Volume=/home/dank/comfyui-data/custom_nodes:/opt/ComfyUI/custom_nodes
 Volume=/home/dank/serverAppConfigs/dankclaw/comfyui/comfyui-healthcheck.py:/usr/local/bin/comfyui-healthcheck.py:ro
 Exec=/opt/ComfyUI/comfyui-start.sh
 # The probe lives in a file because the podman 5.x v1 generator splits
 # HealthCmd on spaces into an argv array; an inline `python3 -c "..."` becomes
 # a puzzle of quoting that fails silently.
 HealthCmd=python3 /usr/local/bin/comfyui-healthcheck.py
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 # First start loads torch and the node graph; give it room before the probe
 # counts against it.
 HealthStartPeriod=300s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,64 +0,0 @@
 {
  "3": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Positive prompt" },
    "inputs": {
      "text": "A detailed illustration of a 1930s Soviet industrial design bureau at night: a well-measured engineer at a drafting table, a warm desk lamp, rolled blueprints, gears and calipers on the desk, deep red and cream palette, gouache painterly style, intricate mechanical detail, soft shadows",
      "clip": [ "6", 0 ]
    }
  },
  "4": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Negative prompt" },
    "inputs": {
      "text": "lowres, blurry, deformed, extra limbs, watermark, signature, jpeg artifacts",
      "clip": [ "6", 0 ]
    }
  },
  "5": {
    "class_type": "UNETLoader",
    "_meta": { "title": "Qwen Image 2.1 (int8)" },
    "inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
  },
  "6": {
    "class_type": "CLIPLoader",
    "_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
    "inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
  },
  "7": {
    "class_type": "VAELoader",
    "_meta": { "title": "VAE" },
    "inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
  },
  "8": {
    "class_type": "EmptySD3LatentImage",
    "_meta": { "title": "Empty Latent 1024x1024" },
    "inputs": { "width": 1024, "height": 1024, "batch_size": 1 }
  },
  "9": {
    "class_type": "KSampler",
    "_meta": { "title": "KSampler" },
    "inputs": {
      "model": [ "5", 0 ],
      "seed": 1930,
      "steps": 20,
      "cfg": 1.0,
      "sampler_name": "euler",
      "scheduler": "simple",
      "positive": [ "3", 0 ],
      "negative": [ "4", 0 ],
      "latent_image": [ "8", 0 ],
      "denoise": 1.0
    }
  },
  "10": {
    "class_type": "VAEDecode",
    "_meta": { "title": "VAE Decode" },
    "inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
  },
  "11": {
    "class_type": "SaveImage",
    "_meta": { "title": "Save Image (art)" },
    "inputs": { "filename_prefix": "qwen21-art", "images": [ "10", 0 ] }
  }
 }
@@ -1,64 +0,0 @@
 {
  "3": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Positive prompt (legible text)" },
    "inputs": {
      "text": "Clean minimalist movie-poster design, dark charcoal background with a subtle vintage grain. Centered large bold white sans-serif text reading \"SOLIDARITY OF THE SOVIET COMPUTER SYSTEMS\", perfectly legible, crisp letterforms, even letter spacing. Below it, a smaller line of yellow text reading \"DANKCLAW DANKSERVER EST 2026\", equally legible. Flat graphic design, high contrast, generous margins, no other words, no watermark",
      "clip": [ "6", 0 ]
    }
  },
  "4": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Negative prompt" },
    "inputs": {
      "text": "blurry, illegible, misspelled, warped letters, extra text, watermark, signature, lowres, jpeg artifacts",
      "clip": [ "6", 0 ]
    }
  },
  "5": {
    "class_type": "UNETLoader",
    "_meta": { "title": "Qwen Image 2.1 (int8)" },
    "inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
  },
  "6": {
    "class_type": "CLIPLoader",
    "_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
    "inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
  },
  "7": {
    "class_type": "VAELoader",
    "_meta": { "title": "VAE" },
    "inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
  },
  "8": {
    "class_type": "EmptySD3LatentImage",
    "_meta": { "title": "Empty Latent 832x1216 poster" },
    "inputs": { "width": 832, "height": 1216, "batch_size": 1 }
  },
  "9": {
    "class_type": "KSampler",
    "_meta": { "title": "KSampler" },
    "inputs": {
      "model": [ "5", 0 ],
      "seed": 2026,
      "steps": 25,
      "cfg": 1.0,
      "sampler_name": "euler",
      "scheduler": "simple",
      "positive": [ "3", 0 ],
      "negative": [ "4", 0 ],
      "latent_image": [ "8", 0 ],
      "denoise": 1.0
    }
  },
  "10": {
    "class_type": "VAEDecode",
    "_meta": { "title": "VAE Decode" },
    "inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
  },
  "11": {
    "class_type": "SaveImage",
    "_meta": { "title": "Save Image (text poster)" },
    "inputs": { "filename_prefix": "qwen21-text", "images": [ "10", 0 ] }
  }
 }
@@ -1,4 +0,0 @@
 # Gitea stack secrets. Copy to /opt/gitea/.env on dankclaw (mode 0600).
 # The live password never enters this repo.
 GITEA_DB_PASSWORD=changeme
@@ -1,31 +0,0 @@
 # Quadlet unit: Gitea PostgreSQL (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/gitea-postgres.container
 # Unit name: gitea-postgres.service (podman 5.x drops the container- prefix)
 #
 # The DB password lives in /home/dank/Containers/gitea/env on the host (mode 0600).
 # This file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gitea.md.
 # The [Service] EnvironmentFile= line lets systemd expand
 # ${GITEA_DB_PASSWORD} in the generated ExecStart.
 [Unit]
 Description=Gitea PostgreSQL container
 After=network.target
 [Container]
 ContainerName=gitea-postgres
 Network=gitea.network
 Image=docker.io/library/postgres:16-alpine
 EnvironmentFile=/home/dank/Containers/gitea/env
 Environment=POSTGRES_USER=gitea
 Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD}
 Environment=POSTGRES_DB=gitea
 Volume=/home/dank/Containers/gitea/postgres:/var/lib/postgresql/data
 [Service]
 EnvironmentFile=/home/dank/Containers/gitea/env
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,44 +0,0 @@
 # Quadlet unit: Gitea web server (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/gitea-server.container
 # Unit name: gitea-server.service (podman 5.x drops the container- prefix)
 #
 # The [Service] EnvironmentFile= line lets systemd expand
 # ${GITEA_DB_PASSWORD} in the generated ExecStart.
 # The DB password lives in /home/dank/Containers/gitea/env on the host (mode 0600).
 # This file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gitea.md.
 #
 # Ports (live): 3000 = HTTP UI/API, 2222 = git SSH. The docs table in
 # Dankclaw Stack.md historically said 3001; the live host port is 3000.
 [Unit]
 Description=Gitea web server container
 Wants=gitea-postgres.service
 After=gitea-postgres.service network.target
 [Container]
 ContainerName=gitea-server
 Network=gitea.network
 Image=docker.io/gitea/gitea:latest
 EnvironmentFile=/home/dank/Containers/gitea/env
 Environment=DATABASE_TYPE=postgres
 Environment=DATABASE_HOST=gitea-postgres
 Environment=DATABASE_NAME=gitea
 Environment=DATABASE_USER=gitea
 Environment=DATABASE_PASSWD=${GITEA_DB_PASSWORD}
 Environment=DATABASE_PORT=5432
 Environment=USER_UID=1000
 Environment=USER_GID=1000
 PublishPort=0.0.0.0:3000:3000
 PublishPort=2222:22
 Volume=/home/dank/Containers/gitea/data:/data
 Volume=/etc/timezone:/etc/timezone:ro
 Volume=/etc/localtime:/etc/localtime:ro
 [Service]
 EnvironmentFile=/home/dank/Containers/gitea/env
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,10 +0,0 @@
 # Quadlet network: private bridge for the Gitea stack (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/gitea.network
 # Unit name: gitea-network.service (podman network name: systemd-gitea)
 #
 # DNS is enabled by default for quadlet networks. This matters: the
 # automatic "podman" default network has dns_enabled=false, so the
 # gitea-postgres and gitea-server container names would not resolve.
 [Network]
@@ -1,100 +0,0 @@
 # Hermes configuration backup / non-secret baseline.
 #
 # Source on Dankclaw: /home/dank/Containers/hermes/data/config.yaml
 # This file is a sanitized Git backup, not a complete replacement for the
 # live Hermes state. Do not add dashboard password hashes, phone numbers,
 # OAuth credentials, API keys, sessions, memories, or generated state here.
 # The live directory is bind-mounted into the containers as /data.
 model:
  provider: dankclaw-large
  base_url: http://192.168.0.143:7770/v1
  api_key: not-needed
  default: frank-dragoon
 providers:
  dankclaw-large:
    api: http://192.168.0.143:7770/v1
    discover_models: false
    models:
      frank-dragoon:
        context_length: 262144
  dankserver-small:
    api: http://192.168.0.235:7770/v1
    discover_models: false
    models:
      frank-medium:
        context_length: 114688
 # Auxiliary routing from the live configuration. Named providers route via
 # their `api:` URL; `custom` entries route by their explicit `base_url`.
 auxiliary:
  vision: {provider: dankclaw-large, model: frank-dragoon}
  web_extract: {provider: dankserver-small, model: frank-medium}
  compression: {provider: dankserver-small, model: frank-medium}
  skills_hub: {provider: dankserver-small, model: frank-medium}
  approval: {provider: dankserver-small, model: frank-medium}
  mcp: {provider: dankserver-small, model: frank-medium}
  # Title generation goes out over the Responses transport on purpose: Hermes
  # hardcodes response_format=json_schema on this one call, no local back-end
  # implements constrained decoding (it is a 400), and this transport is the
  # only one that drops the field. reasoning.effort=none is required with it —
  # the call caps output at 64 tokens, which a thinking pass would consume
  # entirely, leaving an empty title.
  title_generation:
    provider: dankserver-small
    model: frank-medium
    api_mode: codex_responses
    extra_body:
      reasoning:
        effort: none
  memory_query_rewrite: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
  tts_audio_tags: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
  triage_specifier: {provider: dankclaw-large, model: frank-dragoon}
  kanban_decomposer: {provider: dankclaw-large, model: frank-dragoon}
  profile_describer: {provider: dankserver-small, model: frank-medium}
  curator: {provider: dankserver-small, model: frank-medium}
 dashboard:
  # basic_auth intentionally omitted; its password_hash remains live-only.
  font: atkinson-hyperlegible
 moa:
  presets:
    default:
      reference_models:
        - provider: openrouter
          model: deepseek/deepseek-v4-pro
          enabled: false
      degraded_reference_policy: loud
      fanout: user_turn
  reference_models:
    - provider: openrouter
      model: deepseek/deepseek-v4-pro
      enabled: false
  aggregator:
    provider: openrouter
    model: anthropic/claude-opus-4.8
  degraded_reference_policy: loud
  max_tokens: 4096
  fanout: user_turn
  enabled: true
 skills:
  disabled: []
 onboarding:
  seen:
    busy_input_prompt: true
 _config_version: 34
 platforms:
  signal:
    # Live config has the Signal adapter disabled; http_url points at the
    # host signal-cli daemon. The account phone number stays live-only.
    enabled: false
    extra:
      http_url: http://192.168.0.143:8080
 mcp_servers:
  chrome-devtools:
    command: /usr/local/bin/npx
    args:
      - -y
      - chrome-devtools-mcp@latest
      - --browserUrl=http://127.0.0.1:9222
@@ -1,38 +0,0 @@
 # Quadlet unit: Chrome DevTools Protocol daemon for the hermes agent (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/chrome-devtools-protocol.container
 # Unit name: chrome-devtools-protocol.service (container name: chrome-devtools-protocol)
 #
 # Headless Chrome exposing raw CDP over http://127.0.0.1:9222. The hermes
 # agent's chrome-devtools MCP client (mcp_servers.chrome-devtools in
 # /data/config.yaml) spawns npx chrome-devtools-mcp with --browserUrl pointing
 # at this endpoint and drives the browser over it. Both consumers — the Signal
 # gateway and the webui server — run in the host network namespace, so a
 # loopback-only bind is reachable by exactly the processes that may drive the
 # browser and by nobody else on the LAN or WAN.
 #
 # Entrypoint override drops the image's socat front (which would bind
 # 0.0.0.0:9222 in the host namespace and hand browser control to the whole
 # LAN). --no-sandbox is required because rootless podman grants Chrome no user
 # namespace for its sandbox; the content processes therefore run as the uid
 # that already owns this container, capping the blast radius of a hostile page.
 # Image pinned by digest: upgrades are a deliberate unit edit + redeploy, not a
 # tag drift at pull time.
 [Unit]
 Description=Chrome DevTools Protocol daemon (headless-shell) for hermes
 [Container]
 ContainerName=chrome-devtools-protocol
 Network=host
 Image=docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04
 Entrypoint=/headless-shell/headless-shell
 Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222
 ShmSize=1073741824
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,33 +0,0 @@
 # Quadlet unit: Hermes web dashboard (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/hermes-dashboard.container
 # Unit name: hermes-dashboard.service (container name: hermes-dashboard)
 #
 # Management UI for the agent (config, API keys, sessions). Published on
 # 9119 on network dankclaw_default; Caddy's hermesadmin.franksdank.cloud vhost
 # reverse-proxies this port directly (no guard chain) because the dashboard
 # enforces its own basic-auth login from /data/config.yaml
 # (basic_auth.username + scrypt password_hash), making it exempt-by-construction.
 # The name has no public DNS record, so the vhost resolves on the LAN only.
 # Shares /data with the gateway: same agent state, two processes.
 [Unit]
 Description=Hermes web dashboard container
 [Container]
 ContainerName=hermes-dashboard
 Image=localhost/hermes-agent:latest
 Pull=never
 Network=dankclaw_default
 Environment=HERMES_DATA_DIR=/data
 Environment=HERMES_HOME=/data
 Volume=/home/dank/Containers/hermes/data:/data
 PublishPort=9119:9119
 Exec=bash -c 'hermes dashboard --host 0.0.0.0 --port 9119 --no-open --skip-build'
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,65 +0,0 @@
 # Quadlet unit: Hermex webui (Hermex) container (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/hermes-webui.container
 # Unit name: hermes-webui.service (container name: hermes-webui)
 #
 # Runs the open-source hermes-webui (github.com/nesquena/hermes-webui) in
 # foreground mode: bootstrap.py creates .venv inside the mounted checkout,
 # installs WebUI + agent deps, then execs server.py as a single foreground
 # process. It reads HERMES_HOME=/data, so it drives the SAME agent state
 # (memory, skills, sessions) as the gateway container.
 #
 # Host networking on purpose: the agent's chrome-devtools MCP client spawns
 # npx chrome-devtools-mcp with --browserUrl=http://127.0.0.1:9222, and the CDP
 # daemon binds loopback-only in the host network namespace (see
 # chrome-devtools-protocol.container). Only processes sharing that namespace
 # can drive the browser; a bridge-network client could not reach it at all.
 # The server binds 0.0.0.0:8787 of the host netns, and that is deliberate:
 # this unit is host-networked (the CDP daemon above needs a shared loopback), so
 # Caddy cannot reach it by container name and fronts the vhost
 # hermes.franksdank.cloud with `reverse_proxy host.containers.internal:8787`
 # (netavark's host address, 169.254.1.2) — not the host LAN IP, which Caddy's
 # bridge netns cannot reach for a bare host-netns bind; the measured asymmetry
 # with published ports is recorded in the hermes block of dankclaw/Caddyfile.
 # A loopback bind was tried on 2026-09-16 for a tighter posture and broke that
 # vhost, because a bind is also the only way an off-host process can reach a
 # host-networked container: the LAN address and the tunnel's localhost hop are
 # both served by the wildcard, and a loopback-only bind serves only the latter.
 # Controls are the auth chain in front of the vhost plus the password.
 [Unit]
 Description=Hermes webui (Hermex) container
 [Container]
 ContainerName=hermes-webui
 Image=localhost/hermes-agent:latest
 Pull=never
 Network=host
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=HERMES_WEBUI_HOST=0.0.0.0
 Environment=HERMES_WEBUI_PORT=8787
 Environment=HERMES_WEBUI_PASSWORD=${HERMES_WEBUI_PASSWORD}
 Environment=HERMES_HOME=/data
 # Headless host: never show the first-run wizard (a profile already
 # exists in /data from the gateway).
 Environment=HERMES_WEBUI_SKIP_ONBOARDING=1
 Volume=/home/dank/Containers/hermes/data:/data
 Volume=/home/dank/Containers/hermes/webui:/webui
 Exec=python3 /webui/bootstrap.py --foreground
 # JSON exec form on purpose: this quadlet generator joins a bare HealthCmd
 # into ONE argv element with \x20 escapes and then shell-executes it — the
 # python one-liner's quoting would be mangled. A JSON array is passed through
 # verbatim. Start period covers first-boot pip installs into /webui/.venv.
 HealthCmd=["python3","-c","import urllib.request;urllib.request.urlopen('http://127.0.0.1:8787/health')"]
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=5
 HealthStartPeriod=600s
 [Service]
 Restart=always
 RestartSec=5
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 [Install]
 WantedBy=default.target
@@ -1,45 +0,0 @@
 # Quadlet unit: Hermes agent gateway (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/hermes.container
 # Unit name: hermes.service (container name: hermes)
 #
 # The messaging gateway (Signal adapter -> the host signal-cli daemon). The
 # image's default s6 entrypoint would run the interactive TUI, which exits
 # without a TTY; Exec= below replaces it with the headless gateway command,
 # spliced by the v1 quadlet generator as trailing args after the image ref
 # (same splice pattern proven live on authentik-worker). The gateway drops to
 # the unprivileged hermes user (uid 10000) at runtime; /data is world-writable
 # on purpose so that unmapped uid can write agent.log and its state there.
 #
 # Host networking: the adapter dials the signal-cli daemon, a HOST process
 # listening on :8080. The compose era dialed the host LAN IP from a bridge
 # network, where only DNAT'd published ports accept connections to that
 # address — un-forwarded host ports refuse instantly (verified live
 # 2026-08-30). A host-network client dials 127.0.0.1:8080 with no such
 # dependency, and the gateway publishes nothing inbound.
 [Unit]
 Description=Hermes agent gateway container
 After=network-online.target
 Wants=network-online.target
 [Container]
 Network=host
 Image=localhost/hermes-agent:latest
 Pull=never
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 Environment=HERMES_DATA_DIR=/data
 Environment=HERMES_HOME=/data
 Environment=SIGNAL_HTTP_URL=http://127.0.0.1:8080
 Environment=SIGNAL_ACCOUNT=${SIGNAL_ACCOUNT}
 Environment=SIGNAL_ALLOWED_USERS=${SIGNAL_ALLOWED_USERS}
 Volume=/home/dank/Containers/hermes/data:/data
 Exec=bash -c 'hermes gateway run --force -v'
 [Service]
 Restart=always
 RestartSec=5
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
 [Install]
 WantedBy=default.target
@@ -1,43 +0,0 @@
 # If you change this file or any /etc/default/grub.d/*.cfg file,
 # run 'update-grub' afterwards to update /boot/grub/grub.cfg.
 # For full documentation of the options in these files, see:
 #   info -f grub -n 'Simple configuration'
 GRUB_DEFAULT=0
 GRUB_TIMEOUT_STYLE=hidden
 GRUB_TIMEOUT=0
 GRUB_DISTRIBUTOR=`( . /etc/os-release && echo ${NAME} )`
 # Identity (passthrough) DMA since 2026-10-18: the IOMMU stays enabled, so the
 # XDNA2 NPU (/dev/accel/accel0, PASID/SVA) is usable again, while the iGPU's DMA
 # runs untranslated. Decided in serverAppConfigs-docs/LLM/
 # dankclaw-iommu-translation-off.md; install with scripts/set-dankclaw-bootline.sh.
 GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=pt"
 # If your computer has multiple operating systems installed, then you
 # probably want to run os-prober. However, if your computer is a host
 # for guest OSes installed via LVM or raw disk devices, running
 # os-prober can cause damage to those guest OSes as it mounts
 # filesystems to look for things.
 #GRUB_DISABLE_OS_PROBER=false
 # Uncomment to enable BadRAM filtering, modify to suit your needs
 # This works with Linux (no patch required) and with any kernel that obtains
 # the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
 #GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
 # Uncomment to disable graphical terminal
 #GRUB_TERMINAL=console
 # The resolution used on graphical terminal
 # note that you can use only modes which your graphic card supports via VBE/GOP/UGA
 # you can see them in real GRUB with the command `videoinfo'
 #GRUB_GFXMODE=640x480
 # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
 #GRUB_DISABLE_LINUX_UUID=true
 # Uncomment to disable generation of recovery mode menu entries
 #GRUB_DISABLE_RECOVERY="true"
 # Uncomment to get a beep at grub start
 #GRUB_INIT_TUNE="480 440 1"
@@ -1,65 +0,0 @@
 #!/bin/sh
 # halogen-fabric-clock hold | release | status: the AMD GPU's fabric clock (FCLK), held at its top level.
 #
 # On Strix Halo the GPU and the NPU working at the same time can hang the machine, and corrupt the NPU's results,
 # while the GPU's fabric clock changes speed; held at its top level, the two run together cleanly. halogen-flash-server
 # with HALOGEN_NPU_MODELS refuses to start until the clock is held. This holds it: the performance level `manual` and
 # only the top pp_dpm_fclk level, on every amdgpu device that has the control (the GPU and SoC clocks stay automatic).
 # Cost: about 1 W at idle. Run as root; halogen-fabric-clock.service runs `hold` at every boot and after a resume.
 #
 #   hold     hold it (waits up to 60 s for the GPU driver to come up)
 #   release  give it back to the driver (the performance level `auto`)
 #   status   print each device's state; exit 0 when every device is held
 set -u
 SYS="${HALOGEN_SYSFS:-/sys}"
 devices() {
  for d in "$SYS"/class/drm/card*/device; do
    [ -w "$d/pp_dpm_fclk" ] || [ -r "$d/pp_dpm_fclk" ] || continue
    [ -e "$d/power_dpm_force_performance_level" ] || continue
    echo "$d"
  done | sort -u
 }
 held() {   # held DEVICE_DIR: the performance level high, or manual with only the top fclk level selected
  perf=$(cat "$1/power_dpm_force_performance_level" 2>/dev/null)
  [ "$perf" = high ] && return 0
  [ "$perf" = manual ] || return 1
  top=$(grep . "$1/pp_dpm_fclk" | tail -n 1)
  [ "$(grep -c '\*' "$1/pp_dpm_fclk")" = 1 ] && [ "${top%\*}" != "$top" ]
 }
 cmd="${1:-status}"
 case "$cmd" in
  hold)
    n=0
    while [ -z "$(devices)" ] && [ "$n" -lt 60 ]; do sleep 1; n=$((n + 1)); done
    [ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control (pp_dpm_fclk) under $SYS/class/drm" >&2; exit 1; }
    rc=0
    for d in $(devices); do
      lvl=$(grep . "$d/pp_dpm_fclk" | tail -n 1 | cut -d: -f1)
      # the driver applies the level several seconds after the write, so the read-back waits up to 30 s
      ok=0
      if echo manual > "$d/power_dpm_force_performance_level" && echo "$lvl" > "$d/pp_dpm_fclk"; then
        t=0; until held "$d" || [ "$t" -ge 300 ]; do sleep 0.1; t=$((t + 1)); done
        held "$d" && ok=1
      fi
      if [ "$ok" = 1 ]; then
        echo "halogen-fabric-clock: $d: held at level $lvl ($(grep . "$d/pp_dpm_fclk" | tail -n 1 | awk '{print $2}'))"
      else
        echo "halogen-fabric-clock: $d: could not hold it (level $lvl)" >&2; rc=1
      fi
    done
    exit "$rc" ;;
  release)
    for d in $(devices); do echo auto > "$d/power_dpm_force_performance_level" && echo "halogen-fabric-clock: $d: released (auto)"; done ;;
  status)
    [ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control"; exit 1; }
    rc=0
    for d in $(devices); do
      if held "$d"; then echo "$d: held ($(cat "$d/power_dpm_force_performance_level"))"
      else echo "$d: NOT held ($(cat "$d/power_dpm_force_performance_level"))"; rc=1; fi
    done
    exit "$rc" ;;
  *) echo "usage: halogen-fabric-clock hold | release | status" >&2; exit 2 ;;
 esac
@@ -1,19 +0,0 @@
 # Holds the AMD GPU's fabric clock at its top level at boot and after every resume, for halogen-flash-server with
 # the NPU (HALOGEN_NPU_MODELS): on Strix Halo the GPU and the NPU at once are safe only with that clock held.
 # Install (as root), from this directory:
 #   install -m 755 halogen-fabric-clock /usr/local/sbin/
 #   install -m 644 halogen-fabric-clock.service /etc/systemd/system/
 #   systemctl daemon-reload && systemctl enable --now halogen-fabric-clock.service
 # Check: halogen-fabric-clock status. Undo: systemctl disable halogen-fabric-clock.service, then
 #   halogen-fabric-clock release (or reboot). The unit runs once at boot and once after each resume (a resume can hand
 #   the clock back to the driver); it holds nothing open, so it shows inactive between runs.
 [Unit]
 Description=Hold the AMD GPU fabric clock at its top level (GPU and NPU together)
 After=systemd-modules-load.service suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
 [Service]
 Type=oneshot
 ExecStart=/usr/local/sbin/halogen-fabric-clock hold
 [Install]
 WantedBy=multi-user.target suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
@@ -1,30 +0,0 @@
 # Loki + a static busybox.
 #
 # grafana/loki is distroless: no shell, no wget, so nothing can run inside the
 # container to check it. This image drops a static busybox into it for the
 # healthcheck. All Loki behavior is unchanged.
 #
 # The healthcheck lives HERE, in exec form, and not in the Quadlet unit: a
 # unit's HealthCmd reaches podman as CMD-SHELL, which needs /bin/sh — the one
 # thing this image does not have. Specifying it here is also what lets the
 # container report its own health instead of looking merely "Up" while wedged.
 #
 # Build on dankclaw:
 #   podman build --format docker -t franksdank/loki:latest \
 #       -f dankclaw/loki/Containerfile .
 #
 # --format docker is required: podman's default OCI image format has no
 # HEALTHCHECK field, so a default-format build warns and silently discards the
 # one below, leaving a container that reports no health at all.
 FROM docker.io/library/alpine:latest AS tools
 # busybox-static is statically linked: the target images have no dynamic
 # loader (loki is distroless), so the alpine default /bin/busybox (musl
 # dynamic) would fail with "no such file or directory".
 RUN apk add --no-cache busybox-static
 FROM docker.io/grafana/loki:latest
 COPY --from=tools /bin/busybox.static /usr/local/bin/busybox
 # Exec form (JSON array): no shell is involved.
 HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
    CMD ["/usr/local/bin/busybox", "wget", "--spider", "-q", "http://localhost:3100/ready"]
@@ -1,34 +0,0 @@
 # Quadlet unit: Marreta browser companion — Lightpanda (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/marreta-browser.container
 # Unit name: marreta-browser.service (container name: marreta_browser)
 #
 # Lightpanda is a headless browser that exposes the Chrome DevTools Protocol
 # over a WebSocket on port 9222. Marreta drives it to render JS-heavy pages
 # the plain curl fetch cannot. It shares gluetun's network namespace so the
 # browser's OWN outbound fetches also exit the WireGuard tunnel — this is
 # required because marreta's PROXY_URL only covers the curl strategy; the
 # CDP session passes no proxy to the browser, so without netns sharing the
 # browser would leak the host's real egress IP.
 #
 # Marreta reaches it on loopback (same netns): ws://127.0.0.1:9222. No
 # published port — nothing outside the netns talks to it directly.
 # See serverAppConfigs-docs/Dankclaw/Marreta.md.
 [Unit]
 Description=Marreta Lightpanda browser companion container
 Wants=gluetun.service
 After=gluetun.service network.target
 [Container]
 ContainerName=marreta_browser
 Network=gluetun.container
 Image=docker.io/lightpanda/browser:nightly
 [Service]
 TimeoutStartSec=300
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,66 +0,0 @@
 # Quadlet unit: Marreta content-cleaning proxy (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/marreta.container
 # Unit name: marreta.service (container name: marreta)
 #
 # Marreta is a Laravel app (serversideup/php:8.4-fpm-nginx) that strips
 # paywalls/tracking from a fetched page. It reuses gluetun's network
 # namespace (Network=gluetun.container) so ALL of its outbound fetches exit
 # the same WireGuard tunnel as SearXNG — the curl strategy AND, because the
 # browser companion shares the same netns, the Lightpanda fetches too
 # (marreta's PROXY_URL does NOT cover the browser: the CDP connection passes
 # no proxy, so netns sharing is the only way to keep the browser's egress on
 # the tunnel).
 #
 # Port: the shared netns already has SearXNG on 8080, so marreta must NOT
 # bind 8080. NGINX_HTTP_PORT=8085 relocates it (the fpm-nginx image does
 # not bake conf.d/*.conf; the site config is envsubst'd from
 # ${NGINX_HTTP_PORT} at container start). Gluetun publishes host 8082 ->
 # 8085 so Caddy can reach it; see dankclaw/searxng/quadlet/gluetun.container.
 # The app reaches its browser companion over loopback (same netns):
 # ws://127.0.0.1:9222.
 #
 # Secrets (APP_KEY, admin creds) live in /home/dank/Containers/marreta/env
 # (mode 0600, dank-owned so the rootless unit can read it); this file holds
 # none. See serverAppConfigs-docs/Dankclaw/Marreta.md.
 [Unit]
 Description=Marreta content-cleaning proxy container
 Wants=gluetun.service marreta-browser.service
 After=gluetun.service marreta-browser.service network.target
 [Container]
 ContainerName=marreta
 Network=gluetun.container
 Image=ghcr.io/manualdousuario/marreta:latest
 # Relocate off SearXNG's 8080 inside the shared netns.
 Environment=NGINX_HTTP_PORT=8085
 Environment=APP_NAME=Marreta
 Environment=APP_ENV=production
 Environment=APP_DEBUG=false
 Environment=APP_URL=https://marreta.franksdank.cloud
 Environment=APP_LOCALE=en
 Environment="APP_DESCRIPTION=Paywall-free reading"
 Environment=BROWSER_WS_ENDPOINT=ws://127.0.0.1:9222
 Environment=DISABLE_CACHE=false
 # Secrets expanded from the EnvironmentFile below.
 Environment=APP_KEY=${MARRETA_APP_KEY}
 Environment=ADMIN_EMAIL=${MARRETA_ADMIN_EMAIL}
 Environment=ADMIN_PASSWORD=${MARRETA_ADMIN_PASSWORD}
 # Named volume: podman auto-creates it and copyups the image's storage/app
 # ownership (www-data), so the app can write its SQLite DB + cache.
 Volume=marreta_storage:/var/www/html/storage/app
 HealthCmd=["curl","-fsS","-o","/dev/null","http://127.0.0.1:8085/up"]
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=5
 HealthStartPeriod=120s
 [Service]
 EnvironmentFile=/home/dank/Containers/marreta/env
 TimeoutStartSec=900
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,54 +0,0 @@
 # Quadlet unit: alertmanager (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/alertmanager.container
 # Unit name: alertmanager.service (container name: alertmanager)
 #
 # The delivery half of the LLM alert rules. Until 2026-09-17 the rules in
 # shared/prometheus/rules/llm-fleet.yml evaluated and reached nobody — a
 # firing alert was visible in Prometheus and on the Grafana dashboard and
 # nowhere else, and an undelivered alert looks identical to a quiet one.
 # This unit closes that gap: Prometheus pushes to alertmanager:9093 over the
 # dankclaw_default network, and the single receiver webhooks the host-side
 # signal-relay user unit, which hands the text to signal-cli under Frank's
 # existing Signal account.
 #
 # PUBLISHING. 127.0.0.1 only, like every other observability unit: the
 # unauthenticated Alertmanager API (lifecycle endpoints are always on in
 # this binary — there is no flag to gate them) has no business on a host
 # interface, and no off-host consumer needs it. Caddy gains no vhost for
 # it; the homepage timer probes loopback.
 #
 # CONFIG. /opt/alertmanager/config/alertmanager.yml is installed from
 # shared/prometheus/alertmanager.yml by deploy-remote.sh on every deploy, so
 # the repo copy is the source of truth. STATE lives in /opt/alertmanager/data.
 #
 # Rootless: container root maps to the invoking user (dank) in this user
 # namespace and dank owns /opt/alertmanager/data. The image's default user
 # (nobody) maps to a subuid that cannot write it — same User=0:0 as prometheus.
 [Unit]
 Description=Alertmanager alert delivery (Dankclaw)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=alertmanager
 Image=docker.io/prom/alertmanager:latest
 Network=dankclaw_default
 User=0:0
 Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
 PublishPort=127.0.0.1:9093:9093
 Volume=/opt/alertmanager/config:/etc/alertmanager
 Volume=/opt/alertmanager/data:/alertmanager
 HealthCmd=wget --spider -q http://localhost:9093/-/healthy
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,63 +0,0 @@
 # Quadlet unit: alloy (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/alloy.container
 # Unit name: alloy.service (container name: alloy)
 #
 # Log shipper. This replaces the compose service of the same name in
 # dankclaw/compose.yaml ("Observability" block).
 #
 # IMAGE. franksdank/alloy is built locally by deploy-remote.sh
 # (build_compose_image) from dankclaw/alloy/Containerfile: upstream
 # grafana/alloy plus a static busybox for the healthcheck. Pull=never keeps a
 # boot from reaching for that local tag at a registry.
 #
 # NETWORK. It joins dankclaw_default so its config's loki.write endpoint
 # resolves the container name loki:3100.
 #
 # THE UI PORT EXISTS ON PURPOSE. Alloy's UI listens on the container's own
 # 127.0.0.1 by default and cannot be moved from the config file (alloy 1.x
 # refuses the ui block), so the compose service had no reachable health
 # endpoint at all: the homepage status card for alloy reported "down"
 # continuously and there was nothing for a deploy to wait on. The command
 # below therefore moves the UI to the container's wildcard address and the
 # port is published on host loopback only, which puts alloy in the same class
 # as the other three: reachable from the host, by Caddy over the private
 # network, and by nothing off-host. Alloy has no authentication, which is
 # exactly why it is not published wider.
 #
 # SOURCES. The host journald socket and the host log/container-storage trees
 # are bind-mounted read-only; Alloy tails them and pushes to Loki. The config
 # lives at /opt/alloy/config/config.alloy, installed from
 # shared/alloy/config.alloy on every deploy.
 #
 # ORDERING. Wants/After loki.service: without a reachable Loki, Alloy's write
 # client fails and its log pipeline backs up.
 [Unit]
 Description=Alloy log shipper (Dankclaw)
 After=network-online.target loki.service
 Wants=network-online.target loki.service
 [Container]
 ContainerName=alloy
 Image=localhost/franksdank/alloy:latest
 Pull=never
 Network=dankclaw_default
 Exec=run --server.http.listen-addr=0.0.0.0:12345 /etc/alloy/config.alloy
 PublishPort=127.0.0.1:12345:12345
 Volume=/opt/alloy/config:/etc/alloy
 Volume=/run/systemd/journal/socket:/run/systemd/journal/socket:ro
 Volume=/var/log:/var/log:ro
 Volume=/var/lib/containers:/var/lib/containers:ro
 HealthCmd=/usr/local/bin/busybox wget --spider -q http://localhost:12345/-/healthy
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,79 +0,0 @@
 # Quadlet unit: grafana (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/grafana.container
 # Unit name: grafana.service (container name: grafana)
 #
 # Dashboards. This replaces the compose service of the same name in
 # dankclaw/compose.yaml ("Observability" block).
 #
 # PUBLISHING. Bound to 127.0.0.1 for the same reason as prometheus: the browser
 # path is https://grafana.franksdank.cloud (Caddy → Anubis → Authentik outpost →
 # grafana:3000 over dankclaw_default) and the host-side homepage status timer
 # probes 127.0.0.1:3002. Publishing on 0.0.0.0 as the compose service did left
 # an off-host path into Grafana that skips the guard chain entirely.
 #
 # AUTH. Grafana is an OAuth2 relying party against Authentik (OAuth2Provider
 # "grafana", confidential client). The guard chain fronts unauthenticated
 # traffic; the login itself happens through the Authentik button inside
 # Grafana. The client secret is NOT written here: it is expanded by systemd
 # from the root .env (EnvironmentFile in [Service], never in [Container], so
 # the container receives exactly this one secret and not the whole file).
 # Without that .env entry the unit will not start — the same dependency every
 # other rendered-secret deploy has.
 #
 # STATE. /opt/grafana/data holds grafana.db. /opt/grafana/provisioning is
 # mounted read-only and was EMPTY as of 2026-09-11, and grafana.db reported
 # zero datasources and zero dashboards: this Grafana is an empty install whose
 # only configured account is the local admin. Nothing here queries Prometheus
 # or Loki yet.
 [Unit]
 Description=Grafana dashboards (Dankclaw)
 After=network-online.target prometheus.service
 Wants=network-online.target prometheus.service
 [Container]
 ContainerName=grafana
 Image=docker.io/grafana/grafana:latest
 Network=dankclaw_default
 User=0:0
 PublishPort=127.0.0.1:3002:3000
 Environment=GF_SERVER_ROOT_URL=https://grafana.franksdank.cloud
 Environment=GF_AUTH_GENERIC_OAUTH_ENABLED=true
 Environment=GF_AUTH_GENERIC_OAUTH_NAME=Authentik
 Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_ID=grafana-21ca190cde75d60f
 Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=${GRAFANA_OAUTH_CLIENT_SECRET}
 # Global authorize path: this Authentik build resolves the provider by the
 # client_id query parameter — /application/o/<slug>/authorize/ does not exist.
 Environment=GF_AUTH_GENERIC_OAUTH_AUTH_URL=https://auth.franksdank.cloud/application/o/authorize/
 Environment=GF_AUTH_GENERIC_OAUTH_TOKEN_URL=https://auth.franksdank.cloud/application/o/token/
 # ?api=1 = introspection-style userinfo: returns custom claims (groups) too.
 Environment=GF_AUTH_GENERIC_OAUTH_API_URL=https://auth.franksdank.cloud/application/o/userinfo/?api=1
 Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=false
 Environment=GF_AUTH_GENERIC_OAUTH_AUTO_UPSERT=true
 Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_ASSIGN_ORG=true
 Environment=GF_AUTH_GENERIC_OAUTH_ORG_ID=1
 Volume=/opt/grafana/data:/var/lib/grafana
 Volume=/opt/grafana/provisioning:/etc/grafana/provisioning:ro
 HealthCmd=wget --spider -q http://localhost:3000/api/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=60s
 [Service]
 # systemd-side only: supplies ${GRAFANA_OAUTH_CLIENT_SECRET} for the expansion
 # above. The container never receives this file.
 #
 # It is the per-service env file the deployer seeds from dankclaw/.env — the
 # fleet convention (caddy, pihole, copyparty, romm and seerr all read one).
 # The repo-root .env that the compose service happened to pick up is a gluetun
 # env file: it carries HOME=/root and the WireGuard keys, and importing it into
 # this service makes podman look for its configuration under /root, which it
 # cannot read, so the unit fails to start. Seed the value into dankclaw/.env.
 EnvironmentFile=/home/dank/Containers/grafana/env
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,57 +0,0 @@
 # Quadlet unit: loki (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/loki.container
 # Unit name: loki.service (container name: loki)
 #
 # Log store. This replaces the compose service of the same name in
 # dankclaw/compose.yaml ("Observability" block).
 #
 # IMAGE. franksdank/loki is built locally by deploy-remote.sh
 # (build_compose_image) from dankclaw/loki/Containerfile: upstream
 # grafana/loki plus a static busybox, because loki is distroless and the
 # healthcheck has nothing to run inside it. Pull=never keeps a boot from
 # trying to fetch that local tag from a registry. The localhost/ prefix is
 # required — this host's quadlet path cannot resolve a registry-prefixed ref
 # from local storage at create time.
 #
 # PUBLISHING. Bound to 127.0.0.1: Alloy pushes to loki:3100 over
 # dankclaw_default, Caddy proxies the guarded vhost to loki:3100 the same way,
 # and the homepage status timer probes 127.0.0.1:3100. Loki has
 # auth_enabled: false, so the compose service's 0.0.0.0 binding was an
 # unauthenticated log-ingest and read API on every host interface.
 #
 # CONFIG. /opt/loki/config/loki-config.yaml is installed from
 # shared/loki/loki-config.yaml on every deploy (single-binary, filesystem
 # storage, 30-day retention). Chunks and rules live in /opt/loki/data.
 #
 # Rootless: container root maps to the invoking user (dank), who owns
 # /opt/loki/data; the image's default user (10001) maps to a subuid that
 # cannot write it.
 [Unit]
 Description=Loki log store (Dankclaw)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=loki
 Image=localhost/franksdank/loki:latest
 Pull=never
 Network=dankclaw_default
 User=0:0
 Exec=-config.file=/etc/loki/loki-config.yaml
 PublishPort=127.0.0.1:3100:3100
 Volume=/opt/loki/config:/etc/loki
 Volume=/opt/loki/data:/loki
 # No HealthCmd here on purpose. A unit's HealthCmd reaches podman as
 # CMD-SHELL, which means `/bin/sh -c <command>` — and this image is
 # distroless, with no shell at all. The healthcheck therefore ships in the
 # image (dankclaw/loki/Containerfile) in exec form, where it also carries its
 # interval, timeout, start period and retry count.
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,54 +0,0 @@
 # Quadlet unit: node-exporter (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
 # Unit name: node-exporter.service (container name: node-exporter)
 #
 # Host metrics source for Prometheus. This replaces the compose service of the
 # same name in dankclaw/compose.yaml ("Observability" block).
 #
 # HOST NAMESPACE. It must run in the host network namespace to report the
 # host's own interfaces/connections, and --pid=host for the processes and
 # cgroups collectors. "localhost" inside the container is therefore the host's
 # loopback, which is what the homepage status timer probes.
 #
 # LISTEN ADDRESSES. Until 2026-09-11 this bound :9100 on every interface. It
 # now binds exactly two: loopback for the homepage status probe, and the LAN
 # address because Prometheus — which runs in a container on dankclaw_default —
 # scrapes http://192.168.0.143:9100 (shared/prometheus/prometheus.yml,
 # job "node-dankclaw"). node_exporter itself has no authentication, so keeping
 # the wildcard bind off the table matters: this is the second half of the
 # pairing that removes the unauthenticated observability surface, and both
 # consumers are accounted for. Dankserver runs its own copy of this container
 # (scraped by the "node-dankserver" job) — since 2026-09-16 as a Quadlet unit in
 # the same shape (dankserver/observability/quadlet/), because the compose-only
 # version it replaced had been dead for a day and a half before anyone noticed.
 #
 # MOUNTS. The container reads the host's /proc, /sys and / as /host/* read-only
 # and is pointed at them with --path.*; it writes nothing anywhere.
 [Unit]
 Description=Prometheus node exporter (Dankclaw host metrics)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=node-exporter
 Image=docker.io/prom/node-exporter:latest
 Network=host
 PodmanArgs=--pid=host
 Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.143:9100
 Volume=/proc:/host/proc:ro
 Volume=/sys:/host/sys:ro
 Volume=/:/rootfs:ro
 HealthCmd=wget --spider -q http://localhost:9100/metrics
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,64 +0,0 @@
 # Quadlet unit: prometheus (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/prometheus.container
 # Unit name: prometheus.service (container name: prometheus)
 #
 # The homelab metrics store. This replaces the compose service of the same
 # name in dankclaw/compose.yaml ("Observability" block): these five containers
 # were started by hand with podman-compose and never came back after a reboot,
 # which the 2026-09-11 reboot for the amd_iommu=off change made obvious.
 #
 # PUBLISHING. The published port is bound to 127.0.0.1 deliberately. Prometheus
 # is reached by Caddy over the dankclaw_default network (the Caddyfile's
 # post-auth handler proxies to prometheus:9090) and by the host-side homepage
 # status timer over 127.0.0.1. The compose service published on 0.0.0.0, which
 # put an UNAUTHENTICATED metrics API — with the lifecycle endpoint enabled —
 # on every host interface: a bypass around the Anubis + Authentik guard on
 # prometheus.franksdank.cloud. No off-host consumer needs it.
 #
 # CONFIG. /opt/prometheus/config/prometheus.yml is installed from
 # shared/prometheus/prometheus.yml by deploy-remote.sh on every deploy, so the
 # repo copy is the source of truth. STATE (30-day retention) lives in
 # /opt/prometheus/data.
 #
 # Rootless: container root maps to the invoking user (dank) in this user
 # namespace and dank owns /opt/prometheus/data. The image's default user
 # (nobody) maps to a subuid that cannot write it.
 #
 # RULES. Five LLM alert rules and one recording rule load from
 # /etc/prometheus/rules/ (installed from shared/prometheus/rules/ by the
 # deployer). Delivery went live 2026-09-17: the alerting block in
 # prometheus.yml points at alertmanager:9093, which webhooks every firing
 # group to the host-side Signal relay (192.168.0.143:18093, reached from
 # the container as host.containers.internal), which sends through signal-cli account 318311.
 # The Grafana dashboard's ALERTS panel remains the second surface.
 #
 # The config volume is also why rules belong in git: editing them in the UI is
 # lost on the next deploy, since the deployer overwrites the whole directory.
 [Unit]
 Description=Prometheus metrics store (Dankclaw)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=prometheus
 Image=docker.io/prom/prometheus:latest
 Network=dankclaw_default
 User=0:0
 Exec=--config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --storage.tsdb.retention.time=30d --web.enable-lifecycle
 PublishPort=127.0.0.1:9090:9090
 Volume=/opt/prometheus/config:/etc/prometheus
 Volume=/opt/prometheus/data:/prometheus
 HealthCmd=wget --spider -q http://localhost:9090/-/healthy
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,93 +0,0 @@
 # Quadlet unit: open-notebook's SurrealDB (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/open-notebook-surrealdb.container
 # Unit name: open-notebook-surrealdb.service (container name: open-notebook-surrealdb)
 #
 # WHAT IT IS. open-notebook's only datastore: notebooks, notes, sources,
 # provider credentials and podcast episodes all live here. The app talks to it
 # over the WebSocket RPC on the private bridge (SURREAL_URL in
 # scripts/open-notebook.env), which is why the container NAME is part of the
 # contract — renaming it breaks the app's database URL, not just this file.
 #
 # IMAGE. `surrealdb/surrealdb:v2` pinned by manifest-list digest, resolved from
 # the registry on 2026-10-18 (that digest is SurrealDB 2.7.0, image built
 # 2026-09-23; `v2` itself is a MOVING tag, so the digest — not the tag — is what
 # makes this reproducible). Upstream open-notebook's own compose file pins the
 # same `v2` tag, and its Dockerfile copies /surreal out of the same image for
 # the retired single-container variant, so v2 is the version this app is built
 # against. Pull=never + the documented pull step (docs: [[Open Notebook]]):
 #
 #   podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
 #
 # NO PUBLISHED PORT. 8000 is deliberately not published: the app reaches it by
 # name on systemd-open-notebook, and a published 8000 would put the RPC with
 # the root credentials on every interface (upstream publishes 127.0.0.1:8000
 # "purely for local debugging" — a `podman exec` probe is enough here). The
 # binary binds 0.0.0.0:8000 INSIDE its own namespace via the image's
 # SURREAL_BIND env; without that image default it would bind loopback only and
 # the app could never connect, so no --bind flag belongs here.
 #
 # CREDENTIALS, AND WHY THEY ARE NOT IN THE COMMAND LINE. The `surreal start`
 # CLI reads its options from the environment (`--username` is [env:
 # SURREAL_USER=], `--password` is [env: SURREAL_PASS=] — verified against the
 # pinned image's own `start --help`), so the credentials arrive through the two
 # EnvironmentFile lines and the argv stays clean. Passing `--pass` instead —
 # what upstream's compose does — would put the database password in
 # /proc/<pid>/cmdline, readable by any `ps` on the host. Note the two
 # spellings: the DATABASE reads SURREAL_PASS, the APP reads SURREAL_PASSWORD,
 # and scripts/open-notebook.env declares both because they must hold the same
 # value; the deployer writes that pair when it seeds the host secret file.
 #
 # NO USER= OVERRIDE. The image runs as uid 65532 (Chainguard glibc-dynamic
 # base). With a NAMED volume that works unmodified: podman creates the volume's
 # backing directory owned by the container's user (measured with this digest —
 # the host directory lands on the mapped subuid and the store starts clean),
 # which is exactly why upstream's `user: root` line is not repeated here: that
 # line is commented in upstream as being "required for bind mounts", and this
 # unit has none. The consequence is that the DB files are NOT readable by dank
 # without `podman unshare` — accepted for a datastore nobody edits by hand;
 # episode AUDIO is a host directory for the opposite reason (see the app unit).
 #
 # HEALTH. The base image is distroless — measured against this digest, not even
 # `ls` exists (`exec: "ls": executable file not found in $PATH`), so there is no
 # `/bin/sh`, no curl and no wget. SurrealDB ships its own probe instead:
 # `surreal is-ready` connects to ws://localhost:8000 by default, needs no
 # credentials, prints `OK` and exits 0 while the server is up.
 #
 # It must be handed over as a QUADLET-SIDE JSON ARRAY, not a bare command.
 # podman wraps a bare `HealthCmd` string as `CMD-SHELL` (measured on this host's
 # podman 5.7: `--health-cmd "/surreal is-ready"` stores
 # `{"Test":["CMD-SHELL","/surreal is-ready"]}`), and with no `/bin/sh` in the
 # image that probe fails in ~30 ms with empty output on every interval while the
 # database answers perfectly — which is exactly how this service's first deploy
 # died on 2026-10-19 (`did not become healthy in 90s`). The quoted array survives
 # the generator (`--health-cmd "[\"/surreal\",\"is-ready\"]"`) and stores
 # `["CMD","/surreal","is-ready"]`, which execs directly and goes healthy. Do not
 # "simplify" it back to a bare command.
 [Unit]
 Description=SurrealDB (open-notebook datastore)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=open-notebook-surrealdb
 Image=docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
 Pull=never
 Network=open-notebook.network
 EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
 EnvironmentFile=/home/dank/Containers/open-notebook/env
 Exec=start --log info rocksdb:/mydata/mydatabase.db
 Volume=open-notebook-db:/mydata
 HealthCmd=["/surreal","is-ready"]
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=20s
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,105 +0,0 @@
 # Quadlet unit: open-notebook (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/open-notebook.container
 # Unit name: open-notebook.service (container name: open-notebook)
 #
 # WHAT IT IS. The fleet's podcast generator: notes and sources go in, a
 # two-voice (up to four) episode MP3 comes out. One image runs THREE processes
 # under supervisord — the FastAPI API on 5055, the background worker that
 # writes and speaks the episode, and the Next.js UI on 8502 — so this one unit
 # owns the whole app, and the frontend refuses to start until the API answers
 # (the image's own wait-for-api.sh). Runbook: [[Open Notebook]].
 #
 # IMAGE. `lfnovo/open_notebook:v1-latest` pinned by manifest-list digest,
 # resolved from the registry on 2026-10-18; it is the tag upstream's own
 # docker-compose.yml pins for the multi-container shape. Do NOT reach for the
 # `-single` tags: that variant bundles SurrealDB inside the app image, is
 # deprecated upstream, and would give this host two databases. Pull=never, so
 # the pin is fetched by the documented one-time step (docs: [[Open Notebook]]),
 # which the deployer also runs when the image is absent:
 #
 #   podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
 #
 # NETWORKS, in this order on purpose. The generated run line keeps file order,
 # so the DEFAULT ROUTE rides dankclaw_default: that is the network where this
 # host has measured that a bridge container can reach a host-namespace
 # listener through the gateway name (the Prometheus scrape of frank-zealot is
 # the proof — see [[Container Network Namespaces]]), and frank-tts lives there
 # too, so the TTS credential can be `http://frank-tts:7772/v1`. The private
 # systemd-open-notebook bridge is second and carries only the database; its
 # traffic is on-link, so losing the default route costs it nothing.
 #
 # PUBLISHED PORTS, and the 5055 collision. Upstream publishes 8502 and 5055,
 # and 5055 is exactly the port seerr already holds on this host
 # (`PublishPort=127.0.0.1:5055:5055`; `ss -ltn` on Dankclaw 2026-10-18 shows
 # 127.0.0.1:5055 taken). So the API leaves the container on host port 5056.
 # Only 8502 is published on all interfaces, because it is the single upstream
 # the guard chain dials: notebook.franksdank.cloud → anubis → authentik →
 # http://:8017 → 192.168.0.143:8502 (dankclaw/Caddyfile), and this Caddy lives
 # in root's storage, so a published port is the only way it can reach a rootless
 # quadlet. The API needs no outward face at all — the image's own Next.js layer
 # rewrites /api/* to the FastAPI beside it, so one origin serves UI and API —
 # and the fleet's rule is that loopback-only is for services whose user path is
 # the guard chain, which this became on 2026-10-19; 5056 stays published on
 # loopback because the deployer's health gate dials it there. That is also why
 # API_URL in scripts/open-notebook.env names the vhost rather than a port: left
 # to itself the frontend auto-detects `<host-you-opened>:5055`, and the UI loads
 # while every call fails. OPEN_NOTEBOOK_PASSWORD is not made redundant by the
 # chain: it is what keeps the API honest for everything else on this host, and
 # with it unset the API answers anyone, including the unauthenticated
 # episode-audio download.
 #
 # VOLUMES. `/app/data` is a named volume: uploads, the LangGraph SQLite
 # checkpoints and the extraction caches the image keeps under it
 # (UV_CACHE_DIR, HF_HOME) are app state nobody browses. The ONE exception is
 # finished audio, which is bind-mounted out of the container at
 # /home/dank/media/podcasts. That mount is interim and reversible: episode
 # destination is an open human-only decision (Gitea issue #7, label
 # `human-only` — agents must not work it), and until it lands, episodes are
 # ordinary files on the host. DATA_FOLDER is hard-coded `./data` in the app
 # (WORKDIR /app) and the DB stores audio paths RELATIVE to data/podcasts, so
 # covering that one subdirectory is enough and removing the line changes
 # nothing else. The image declares no USER, so the app runs as container root,
 # which in this host's rootless mapping (0 -> 1000, measured on this host —
 # see [[Seerr]]) writes those episode files as dank.
 #
 # HEALTH. `GET /health` on the API returns {"status":"healthy"} and is in the
 # password middleware's excluded-path list, so the probe stays honest after the
 # password is set (both read from api/main.py upstream). The image installs
 # curl and ffmpeg in the runtime layer, so the probe needs no shell — kept
 # argument-only for the generator's space-splitting. Note what this probe does
 # NOT prove: the route returns a static body, so it says the API serves, not
 # that the database answers. The DB has its own probe in its unit, and the
 # worker's failure to reach the database shows up as failed jobs, not as an
 # unhealthy container.
 [Unit]
 Description=open-notebook (podcast generator: API, worker, web UI)
 After=network-online.target open-notebook-surrealdb.service
 Wants=network-online.target open-notebook-surrealdb.service
 [Container]
 ContainerName=open-notebook
 Image=docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
 Pull=never
 Network=dankclaw_default
 Network=open-notebook.network
 PublishPort=0.0.0.0:8502:8502
 PublishPort=127.0.0.1:5056:5055
 EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
 EnvironmentFile=/home/dank/Containers/open-notebook/env
 Volume=open-notebook-data:/app/data
 Volume=/home/dank/media/podcasts:/app/data/podcasts
 HealthCmd=curl -fsS http://127.0.0.1:5055/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=60s
 [Service]
 Restart=always
 RestartSec=10
 TimeoutStartSec=900
 [Install]
 WantedBy=default.target
@@ -1,34 +0,0 @@
 # Quadlet network: private bridge for the open-notebook pair (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/open-notebook.network
 # Unit name: open-notebook-network.service (podman network name:
 # systemd-open-notebook — quadlet prefixes "systemd-", so probe the runtime
 # name, never the filename; see [[Container Network Namespaces]]).
 #
 # WHY A PRIVATE BRIDGE. The app reaches its database by container name
 # (SURREAL_URL=ws://open-notebook-surrealdb:8000/rpc), and container DNS names
 # resolve only on the network that owns them. Putting SurrealDB here instead of
 # on dankclaw_default keeps the datastore reachable ONLY to the container that
 # owns it: 8000 is not published (an unpublished port is refused outright from
 # every other namespace anyway), and nothing else on the fleet-wide network can
 # dial it with the root credentials. Same reasoning as the gitea and authentik
 # database pairs.
 #
 # Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
 # per-container userspace helper left a container that accepted TCP and reset
 # every request for two days, logging nothing). A bridge has no such helper.
 #
 # Subnet registry on dankclaw (rootless):
 #   10.88.0.0/16  default "podman" bridge
 #   10.89.0.0/24  gitea (systemd-gitea)
 #   10.89.1.0/24  compose stack (dankclaw_default)
 #   10.89.2.0/24  searxng (systemd-searxng)
 #   10.89.3.0/24  open-webui (systemd-open-webui)
 #   10.89.4.0/24  open-notebook (systemd-open-notebook)  <- this file
 # Pinned so the container IPs stay deterministic across reboots, like the
 # other pinned nets; 10.89.4.0/24 was the next unclaimed /24 in this registry.
 [Network]
 Driver=bridge
 Subnet=10.89.4.0/24
 Gateway=10.89.4.1
@@ -1,96 +0,0 @@
 # Quadlet unit: open-webui (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/open-webui.container
 # Unit name: open-webui.service (container name: open-webui)
 #
 # Chat UI for the LLM fleet. It runs on the CPU image (dankclaw is CPU-only, so
 # the :cuda image would waste a large pull for a GPU it cannot use) and points
 # its OpenAI connection at the live llama.cpp backend on this host via the
 # host LAN IP (192.168.0.143:7770, model frank-default-large). It publishes 3005
 # (3000 is Gitea's HTTP port). Data persists under
 # /home/dank/Containers/open-webui/data. See serverAppConfigs-docs/Dankclaw/OpenWebUI.md.
 #
 # Network history (2026-08-22): this unit used Network=slirp4netns. When the
 # slirp4netns helper process died, the container kept running but lost its tap
 # interface: a zombie that accepted TCP and reset every request, and chat
 # served empty responses for ~2 days with no error anywhere. It now runs on a
 # pinned bridge network (open-webui.network), which has no per-container
 # helper to die. The LLM is reached via the host LAN IP (192.168.0.143):
 # the bridge gateway lives in the rootless netns, not the host netns where
 # llama-server runs. The healthcheck probes /health (app liveness) and
 # requires a non-loopback interface (network liveness).
 [Unit]
 Description=open-webui chat UI container
 After=network.target
 [Container]
 ContainerName=open-webui
 Network=open-webui.network
 Image=ghcr.io/open-webui/open-webui:main
 PublishPort=3005:8080
 Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
 Environment=ANONYMIZED_TELEMETRY=False
 Environment=DO_NOT_TRACK=True
 # SSO via authentik (OIDC authorization-code flow, 2026-09-24). The OAuth
 # client (application slug "open-webui") lives in authentik; the client
 # SECRET stays off-repo in the git-ignored host env file (mode 600). The
 # container reaches the IdP over the public URL: Caddy publishes 0.0.0.0:443
 # and Pi-hole resolves auth.franksdank.cloud to the host LAN IP, so the
 # server-side code exchange works from this bridge. Roles come from the
 # "Open WebUI roles" scope mapping (entitlements "Open WebUI Users" /
 # "Open WebUI Admins"); OAUTH_MERGE_ACCOUNTS_BY_EMAIL folds SSO logins into
 # the existing local account (frank8465@protonmail.com). The OAuth vars are
 # read directly from the environment at runtime (the per-key config DB holds
 # only Admin-Panel overrides, so nothing is seeded on the live data dir).
 # NOTE: the OAUTH_SCOPES value MUST stay quoted — systemd splits unquoted
 # Environment= values on whitespace (unquoted, only "openid" survived).
 EnvironmentFile=/home/dank/Containers/open-webui/env
 Environment=OAUTH_CLIENT_ID=KmtzKc0bjWjaIFeM0puYudI1CLXR2p0jnPCSf4N0
 Environment=OAUTH_PROVIDER_NAME=authentik
 Environment=OPENID_PROVIDER_URL=https://auth.franksdank.cloud/application/o/open-webui/.well-known/openid-configuration
 Environment=OPENID_REDIRECT_URI=https://ai.franksdank.cloud/oauth/oidc/callback
 Environment=WEBUI_URL=https://ai.franksdank.cloud
 Environment=ENABLE_OAUTH_SIGNUP=true
 Environment=OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true
 Environment="OAUTH_SCOPES=openid email profile roles"
 Environment=ENABLE_OAUTH_ROLE_MANAGEMENT=true
 # SSO-only (2026-09-24): the OIDC round-trip is verified, so the local
 # username/password path is retired. ENABLE_LOGIN_FORM hides the form;
 # ENABLE_PASSWORD_AUTH refuses password logins entirely, so the only way in
 # is the authentik OIDC flow (which folds into the local account by email).
 # The per-key config DB override (ui.enable_login_form) is flipped to false
 # in the same change — the DB wins over env, so both must agree.
 Environment=ENABLE_LOGIN_FORM=false
 Environment=ENABLE_PASSWORD_AUTH=false
 # Web search -> SearXNG, reached over the host-published port (NOT the
 # search.franksdank.cloud vhost): the Anubis->Authentik guard chain is a
 # browser mechanism and a plain HTTP client cannot solve it. The internal
 # port bypasses Caddy entirely. These env vars SEED the per-key config DB on
 # first run (fill-missing only); once the keys exist in /app/backend/data the
 # DB is authoritative, so this block documents the intent and makes a fresh
 # data dir come up wired. SearXNG's limiter exempts this source (the bridge
 # egress 10.89.x is in pass_ip), so the 4-req/IP/hour JSON cap never applies.
 Environment=ENABLE_WEB_SEARCH=True
 Environment=WEB_SEARCH_ENGINE=searxng
 Environment=SEARXNG_QUERY_URL=http://192.168.0.143:8081/search
 Environment=SEARXNG_LANGUAGE=all
 Environment=WEB_SEARCH_RESULT_COUNT=6
 Volume=/home/dank/Containers/open-webui/data:/app/backend/data
 # App liveness (/health) AND network liveness (a non-loopback interface must
 # exist; a dead tap leaves only "lo" behind).
 HealthCmd=CMD-SHELL python -c 'import os,urllib.request,sys; urllib.request.urlopen("http://127.0.0.1:8080/health",timeout=5).read(); sys.exit(0 if any(i!="lo" for i in os.listdir("/sys/class/net")) else 1)'
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=300s
 [Service]
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,31 +0,0 @@
 # Quadlet network: private bridge for open-webui (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/open-webui.network
 # Unit name: open-webui-network.service (podman network name: systemd-open-webui)
 #
 # Why a bridge instead of slirp4netns (2026-08-22 incident):
 # slirp4netns runs a user-space helper process per container. When that
 # helper died, the container kept running but lost its tap interface,
 # becoming a zombie that accepted TCP and reset every request. chat
 # served empty responses for ~2 days and nothing logged an error.
 # A kernel bridge has no per-container helper, so the failure mode
 # cannot occur.
 #
 # The subnet is pinned so the container's own IP (10.89.3.2) is
 # deterministic. Note the LLM backend is NOT reached via this gateway:
 # the gateway lives in the rootless netns, while llama-server runs in
 # the host netns. The host's LAN IP (192.168.0.143) reaches it through
 # the rootless NAT. host.containers.internal is a slirp4netns-only name
 # and does not resolve on a bridge.
 #
 # Subnet registry on dankclaw (rootless):
 #   10.88.0.0/16  default "podman" bridge
 #   10.89.0.0/24  gitea (systemd-gitea)
 #   10.89.1.0/24  compose stack (dankclaw_default)
 #   10.89.2.0/24  searxng (systemd-searxng)
 #   10.89.3.0/24  open-webui (systemd-open-webui)  <- this file
 [Network]
 Driver=bridge
 Subnet=10.89.3.0/24
 Gateway=10.89.3.1
@@ -1,48 +0,0 @@
 # A records for the flat franksdank.name scheme (dnsmasq addn-hosts format:
 # <IP> <name> [<name> ...]). Served by the Pi-hole quadlet (FTL is a
 # dnsmasq fork): the quadlet mounts this file into /etc/pihole/hosts/ read-only.
 #
 # Caddy is consolidated on dankclaw (192.168.0.143): it fronts every service
 # on both hosts, so every name resolves to that one IP. Keep in sync with the
 # Caddyfile (tests/pihole.bats cross-checks this).
 #
 #   192.168.0.143 = dankclaw (runs the Caddy container + the AI/dev stack)
 #   192.168.0.235 = dankserver (media stack; reached by Caddy via published
 #                    LAN ports — no DNS records point here)
 # --- Homepage (naked domain + www) + legacy names ---
 192.168.0.143 franksdank.cloud
 192.168.0.143 www.franksdank.cloud
 192.168.0.143 dankclaw.franksdank.cloud
 192.168.0.143 dankserver.franksdank.cloud
 # --- Authentik (SSO / OIDC provider) ---
 192.168.0.143 auth.franksdank.cloud
 # --- Media services (dankserver, proxied) ---
 192.168.0.143 plex.franksdank.cloud
 192.168.0.143 plex.franksdank.casa
 192.168.0.143 books.franksdank.cloud
 192.168.0.143 romm.franksdank.cloud
 192.168.0.143 files.franksdank.cloud
 192.168.0.143 jelly.franksdank.cloud
 192.168.0.143 radarr.franksdank.cloud
 192.168.0.143 lidarr.franksdank.cloud
 192.168.0.143 readarr.franksdank.cloud
 192.168.0.143 prowlarr.franksdank.cloud
 192.168.0.143 qbittorrent.franksdank.cloud
 # --- AI / search / dev services (dankclaw, local) ---
 192.168.0.143 search.franksdank.cloud
 192.168.0.143 comfyui.franksdank.cloud
 192.168.0.143 ai.franksdank.cloud
 192.168.0.143 hermes.franksdank.cloud
 192.168.0.143 hermesadmin.franksdank.cloud
 192.168.0.143 gitea.franksdank.cloud
 192.168.0.143 grafana.franksdank.cloud
 192.168.0.143 prometheus.franksdank.cloud
 192.168.0.143 loki.franksdank.cloud
 192.168.0.143 pihole.franksdank.cloud
 192.168.0.143 seerr.franksdank.cloud
 192.168.0.143 marreta.franksdank.cloud
 192.168.0.143 notebook.franksdank.cloud
@@ -1,18 +0,0 @@
 # Pi-hole container environment (dankclaw).
 # Copy to /home/dank/Containers/pihole/env and fill real values.
 # The real env file is never committed (it holds the web UI password).
 # The host IP the container answers for (also shown in the web UI).
 ServerIP=192.168.0.143
 # Timezone for the web UI and logs.
 TZ=America/Los_Angeles
 # Pinned upstream resolvers. The slirp netns has no useful resolv.conf, so
 # pin them (mirrors the retired dnsmasq.conf server= lines).
 DNS1=1.1.1.1
 DNS2=1.0.0.1
 # Web UI password (the shared family password; the same credential as
 # CADDY_BASICAUTH_HASH in the host .env).
 WEBPASSWORD=changeme
@@ -1,42 +0,0 @@
 # Quadlet unit: Pi-hole LAN DNS (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/pihole.container
 # Unit name: pihole.service (container name: pihole)
 #
 # The LAN's authoritative DNS for the flat franksdank.cloud / franksdank.casa
 # name scheme. It retired the dnsmasq quadlet (2026-08-24): Pi-hole FTL is a
 # dnsmasq fork, so the addn-hosts record file carries over unchanged and is
 # mounted into /etc/pihole/hosts/ (the FTL hostsdir) read-only. It serves one A
 # record per Caddy vhost and forwards everything else to the pinned upstream
 # resolvers (DNS1/DNS2 in the env file). The SURFboard G36 router hands this
 # server out to DHCP clients (DNS statically configured -> 192.168.0.143),
 # so no client needs per-device config.
 #
 # Runs on slirp4netns with port 53 (DNS) and 8083 (web UI) published to the
 # LAN (the same pattern as open-webui). Config + state mount from
 # /home/dank/Containers/pihole. The web UI password is WEBPASSWORD in the env
 # file. See serverAppConfigs-docs/Infrastructure/LAN-DNS.md.
 [Unit]
 Description=Pi-hole LAN DNS container
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=pihole
 Image=docker.io/pihole/pihole:latest
 Network=slirp4netns
 PublishPort=0.0.0.0:53:53/udp
 PublishPort=0.0.0.0:53:53/tcp
 PublishPort=0.0.0.0:8083:80/tcp
 EnvironmentFile=/home/dank/Containers/pihole/env
 Volume=/home/dank/Containers/pihole/etc-pihole:/etc/pihole
 Volume=/home/dank/Containers/pihole/etc-dockers:/etc/dockers
 Volume=/home/dank/Containers/pihole/addn-hosts:/etc/pihole/hosts/addn-hosts:ro
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,8 +0,0 @@
 # Rootless DNS must bind port 53: lower the unprivileged-port threshold so a
 # rootless container may publish 0.0.0.0:53 (80/443 stay allowed; 53 < 80).
 # Single source of truth for net.ipv4.ip_unprivileged_port_start on dankclaw;
 # deploy_pihole installs it to /etc/sysctl.d/ and applies it with
 # `sysctl --system` (the same code path boot uses via systemd-sysctl).
 # A later-sorted drop-in that reset this to 80 took LAN DNS down after the
 # 2026-08-26 reboot — deploy_pihole neutralizes such conflicts.
 net.ipv4.ip_unprivileged_port_start=53
@@ -1,40 +0,0 @@
 # Quadlet unit: Gluetun VPN tunnel (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/gluetun.container
 # Unit name: gluetun.service (container name: gluetun)
 #
 # Creates the WireGuard tunnel to ProtonVPN (US-WA). SearXNG reuses this
 # container's network namespace (see searxng.container), so its outbound search
 # traffic exits here. The WireGuard credentials live in /opt/searxng/.env
 # (mode 0600); this file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gluetun.md.
 [Unit]
 Description=Gluetun VPN tunnel container
 Wants=searxng-network.service
 After=searxng-network.service network.target
 [Container]
 ContainerName=gluetun
 Network=searxng.network
 Image=docker.io/qmcgaw/gluetun:latest
 AddCapability=NET_ADMIN
 AddDevice=/dev/net/tun:/dev/net/tun
 Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
 Environment=VPN_TYPE=${VPN_TYPE}
 Environment=WIREGUARD_PRIVATE_KEY=${WIREGUARD_PRIVATE_KEY}
 Environment=WIREGUARD_ADDRESSES=${WIREGUARD_ADDRESSES}
 Environment=WIREGUARD_PUBLIC_KEY=${WIREGUARD_PUBLIC_KEY}
 Environment=WIREGUARD_ENDPOINT_IP=${WIREGUARD_ENDPOINT_IP}
 Environment=WIREGUARD_ENDPOINT_PORT=${WIREGUARD_ENDPOINT_PORT}
 Environment=WIREGUARD_ALLOWED_IPS=${WIREGUARD_ALLOWED_IPS}
 Environment=WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=${WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL}
 PublishPort=8081:8080
 PublishPort=8082:8085
 [Service]
 EnvironmentFile=/opt/searxng/.env
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,28 +0,0 @@
 # Quadlet unit: Valkey cache for SearXNG (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/searxng-valkey.container
 # Unit name: searxng-valkey.service (container name: searxng-valkey)
 #
 # The limiter's rate-limit state store. Joins the searxng network and is
 # reachable by the name "valkey" (settings.yml points there). Data persists in
 # /opt/searxng/valkey. See serverAppConfigs-docs/Dankclaw/Valkey.md.
 [Unit]
 Description=SearXNG Valkey cache container
 Wants=searxng-network.service
 After=searxng-network.service network.target
 [Container]
 ContainerName=searxng-valkey
 Network=searxng.network
 NetworkAlias=valkey
 Image=docker.io/valkey/valkey:9-alpine
 Volume=/opt/searxng/valkey:/data
 [Service]
 EnvironmentFile=/opt/searxng/.env
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,30 +0,0 @@
 # Quadlet unit: SearXNG search engine (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/searxng.container
 # Unit name: searxng.service (container name: searxng)
 #
 # Reuses gluetun's network namespace (Network=gluetun.container) so its
 # outbound traffic exits the WireGuard tunnel. It publishes no port of its own;
 # the port lives on gluetun. It reaches valkey by name on the searxng network.
 # Config mounts from /opt/searxng/config. See serverAppConfigs-docs/Dankclaw/SearXNG.md.
 [Unit]
 Description=SearXNG search engine container
 Wants=gluetun.service searxng-valkey.service
 After=gluetun.service searxng-valkey.service network.target
 [Container]
 ContainerName=searxng
 Network=gluetun.container
 Image=ghcr.io/searxng/searxng:latest
 Environment=SEARXNG_SECRET=${SEARXNG_SECRET}
 Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL}
 Volume=/opt/searxng/config:/etc/searxng
 [Service]
 EnvironmentFile=/opt/searxng/.env
 TimeoutStartSec=900
 Restart=always
 [Install]
 WantedBy=default.target
@@ -1,16 +0,0 @@
 # Quadlet network: private bridge for the SearXNG stack (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/searxng.network
 # Unit name: searxng-network.service (podman network name: systemd-searxng)
 #
 # DNS is enabled by default for quadlet networks. This matters: the automatic
 # "podman" default network has dns_enabled=false, so container names would not
 # resolve. searxng (which reuses gluetun's netns) reaches valkey by the name
 # "valkey" on this network.
 #
 # The subnet is pinned so the value stays stable across reboots. It must match
 # an entry in shared/searxng/limiter.toml trusted_proxies. 10.89.2.0/24 is free
 # (gitea uses 10.89.0.0/24; the old compose net used 10.89.1.0/24).
 [Network]
 Subnet=10.89.2.0/24
@@ -1,43 +0,0 @@
 # Quadlet unit: Seerr media request manager (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/seerr.container
 # Unit name: seerr.service (container name: seerr)
 #
 # Seerr is the unified Overseerr/Jellyseerr successor. It fronts the
 # *arr stack now living on FrankBook-Pro (192.168.0.112, resolved via the
 # Pi-hole addn-hosts record frankbook-pro.lan): requests are placed here,
 # routed through the arrs over the LAN, and downloads run inside the Mac's
 # gluetun tunnel. SQLite lives under /home/dank/Containers/seerr/data —
 # no separate database unit (decision D8).
 #
 # Exposure contract: the ONLY published port is loopback (127.0.0.1:5055)
 # for the homepage probe and host-local checks. User traffic must traverse
 # the guard chain: Caddy TLS edge → Anubis :8923 → Authentik outpost →
 # Caddy :8017 post-auth listener → seerr:5055 on dankclaw_default. Never
 # publish 5055 on a LAN or public address.
 #
 # Image pin: ghcr.io/seerr-team/seerr:v3.4.1 (manifest-verified on GHCR
 # 2026-09-01; released 2026-07-30). Upgrade = bump the tag, redeploy.
 #
 # No secrets in this file: OIDC (Authentik) and arr API keys live in
 # Seerr's own settings store inside the data volume, configured at runtime.
 [Unit]
 Description=Seerr media request manager container
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=seerr
 Image=ghcr.io/seerr-team/seerr:v3.4.1
 Network=dankclaw_default
 PublishPort=127.0.0.1:5055:5055
 Volume=/home/dank/Containers/seerr/data:/app/config
 [Service]
 Restart=always
 RestartSec=5
 TimeoutStartSec=900
 [Install]
 WantedBy=default.target
@@ -1,21 +0,0 @@
 # /etc/udev/rules.d/99-comfyui-gpu-access.rules
 #
 # Why world-writable compute nodes: the comfyui container is spawned by a
 # systemd user unit (comfyui.service, Type=oneshot). User-unit processes are
 # spawned by the user manager with NO supplementary groups, so any spawn path
 # through systemd loses `render`/`video` membership regardless of how the
 # container is launched (keep-groups and named group_add both depend on the
 # spawning process already holding the group — verified 2026-08-31: keep-groups
 # preserved an empty group list under the user manager and /dev/kfd open still
 # failed with EACCES). A mode that grants access to "everyone in this uid
 # domain" is the only spawn-context-independent option short of running the
 # stack rootful.
 #
 # Blast radius: any local process (every container here runs as the same uid
 # domain) can already ptrace the GPU userspace through an existing holder's
 # fd inheritance; on a private single-administrator home server this trades a
 # capability gate that could never be held by the intended user anyway.
 # Note the missing SUBSYSTEM= guard on the kfd line: misc-class devices have no
 # subsystem for udev matching, so a SUBSYSTEM=="misc" guard never matches.
 KERNEL=="kfd", MODE="0666"
 KERNEL=="renderD*", SUBSYSTEM=="drm", MODE="0666"
@@ -1,43 +0,0 @@
 # frank-tts — the cloned-voice speech façade (dankclaw/voiceservice/app.py).
 #
 # Standard library only, so there is nothing to pip-install and nothing to
 # compile: the image is a python runtime plus ffmpeg (the one binary the app
 # shells out to, for response_format=mp3) plus one COPY of app.py. Rebuild to
 # ship a new app.py.
 #
 # Build on dankclaw (deploy-remote.sh build_compose_image frank-tts does this,
 # including --format docker, which the fleet's other local builds also pass):
 #   podman build --format docker -t franksdank/frank-tts:latest \
 #       -f dankclaw/voiceservice/Containerfile .
 #
 # BASE PINNED BY DIGEST, like every other image this fleet runs from a
 # registry: a tag moves under a rebuild and a voice endpoint that changes
 # python or ffmpeg mid-week is a debug session nobody asked for. The digest is
 # the multi-arch INDEX of python:3.12-alpine as published 2026-10-18 (amd64 +
 # arm64 on linux); Dankclaw resolves the amd64 entry.
 #
 # WHY 3.12: the fleet's ceiling for this app is python <= 3.12 (single file,
 # no build step); the alpine flavour also keeps the image in the tens of MB
 # and ships busybox `wget`, which is what the quadlet unit's HealthCmd runs —
 # so no HEALTHCHECK belongs here (the probe lives in the unit, as in
 # alertmanager/prometheus; only distroless bases need it baked in).
 FROM docker.io/library/python:3.12-alpine@sha256:1b668429b3511ab407d8e00648891631b0b1a4d7e15e3ca70f38ab5b91ad4ab4
 # ffmpeg: mp3 output. The app always asks the upstream for wav and re-encodes
 # on the way out, so ffmpeg is the only external program in the picture.
 RUN apk add --no-cache ffmpeg
 # Unbuffered stdout: the unit's journal must show the "listening on" line (and
 # any backend error) the moment it happens, not when the buffer fills.
 ENV PYTHONUNBUFFERED=1
 COPY dankclaw/voiceservice/app.py /app/app.py
 # Non-root. The voice packs are bind-mounted read-only, so this user only ever
 # reads them — they must be world-readable (the pack builder writes 644/755).
 # The one thing it writes is its per-request temp dir under /tmp.
 RUN adduser -D -u 10000 -h /app -s /sbin/nologin shim
 USER shim
 EXPOSE 7772
 CMD ["python3", "/app/app.py"]
@@ -1,178 +0,0 @@
 # frank-tts-engine — the arm A TTS engine: qwentts.cpp's resident `tts-server`
 # (the CPU binary that frank-tts on :7772 forwards to). The runbook is
 # [[Frank TTS Engine]]; the fleet-facing endpoint is [[Frank TTS]].
 #
 # WHAT THIS IS. The upstream source is NOT vendored here: the image clones
 # ServeurpersoCom/qwentts.cpp at one pinned commit and builds it. Rebuilding is
 # therefore a network operation (github.com plus objects.githubusercontent.com
 # for the `ggml` submodule), and build_compose_image in scripts/deploy-remote.sh
 # skips it whenever this Containerfile's sha256 label already matches the built
 # image — the same offline-safe idiom as loki/alloy/comfyui, so a re-deploy of
 # an unchanged engine needs no network.
 #
 # THE PINNED COMMIT IS THE CONTRACT. 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
 # is the commit the 2026-10-18 CPU trial on Dankclaw measured (server-side TTFA
 # 94-112 ms, warm RTF 0.55-0.67, run spread +-0.01 — the numbers that won the
 # engine decision). Moving it means re-measuring, not just editing this line.
 # `ggml` is a submodule of the repo (the only one; vendor/cpp-httplib and
 # vendor/yyjson are ordinary directories in the tree), and the submodule commit
 # is recorded in the parent tree, so cloning the parent commit recursively is
 # reproducible without a second pin.
 #
 # THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim — with the Vulkan door
 # open since 2026-10-19. That configure was run in this exact base image and
 # answers `Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version
 # "1.3.275") found components: glslc` then `Including Vulkan backend`:
 #
 #   cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
 #       -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
 #       -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON \
 #       -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
 #   && cmake --build build -j 8
 #
 # Every flag is load-bearing, so do not "tidy" it:
 #   GGML_BLAS=OFF is FORCED, not a preference — Dankclaw has libopenblas.so.0
 #     but no BLAS dev headers, so the upstream default (BLAS on) fails to
 #     configure. Upstream's own Dockerfile and buildcpu.sh both pass BLAS on,
 #     which is why this file does not copy them.
 #   GGML_CCACHE=OFF because a container build layer has no persistent ccache
 #     directory to hit; on, it only writes a useless one.
 #   GGML_OPENMP=ON is what makes the runtime need libgomp1 (installed below).
 #   GGML_BACKEND_DL=OFF / GGML_CPU_ALL_VARIANTS=OFF: upstream's images build
 #     every x86 ISA as a loadable backend so one image runs anywhere. This one
 #     is built on the machine that runs it, so the single native CPU backend is
 #     what was measured. That is also why no -DGGML_NATIVE is passed: ggml
 #     defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
 #     host — correct while build host == run host, wrong if anyone ever builds
 #     this image elsewhere, which is why the fleet builds it on Dankclaw.
 #   CUDA/HIP/METAL/SYCL/RPC stay OFF, VULKAN is ON: the endpoint computes on the
 #     gfx1151 iGPU (decision and co-tenancy gates:
 #     report/tts-npu-spike-verdict-2026-10-19.md, shape in [[Frank TTS Engine]]).
 #     Nothing else is available to this image: no ROCm userland, no NVIDIA
 #     userland, no macOS, no oneAPI, and RPC would add a second process to
 #     supervise on a box that already has one engine per card.
 #
 # BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
 # tts-server links libggml*.so rather than swallowing them; the runtime stage
 # copies the binary and its shared objects out of the build tree and points
 # LD_LIBRARY_PATH at them, exactly as upstream's own Dockerfile does (the RPATH
 # baked at link time names the build directory, which does not exist here).
 #
 # BASE PINNED BY DIGEST, like every other image this fleet builds: the digest
 # is the multi-arch INDEX of ubuntu:24.04 as resolved from Docker Hub on
 # 2026-10-18; Dankclaw takes the amd64 entry. One base for both stages so there
 # is exactly one thing to re-resolve.
 #
 # NO HEALTHCHECK IN THIS FILE. The probe lives in the Quadlet unit
 # (HealthCmd=), which is the fleet pattern for anything with a shell — see the
 # alertmanager/prometheus units, and the opposite case in dankclaw/loki, whose
 # distroless image has no /bin/sh for the generator's CMD-SHELL wrapping.
 # `curl` is therefore installed in the runtime stage, for that probe only.
 ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac
 # ------------------------------------------------------------------ build
 FROM ${BASE_IMAGE} AS build
 # The three Vulkan build deps are not optional and not interchangeable:
 #   libvulkan-dev    vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake
 #                    configs, i.e. what find_package(Vulkan) resolves to.
 #   glslc            ggml compiles its shaders with glslc, not glslangValidator.
 #                    Ubuntu's glslang-tools ships glslangValidator and NOT glslc,
 #                    so without this package `find_package(Vulkan COMPONENTS
 #                    glslc REQUIRED)` fails and there is no Vulkan backend.
 #                    The archive's glslc is old (2023.8): it compiles
 #                    GL_KHR_cooperative_matrix — the path ggml's quantized
 #                    matmul needs — but reports cooperative_matrix2,
 #                    integer_dot_product, bfloat16 and the micro-float
 #                    extensions as unsupported, so those shader families are
 #                    compiled out. If a Vulkan prefill ever reads as
 #                    inexplicably slow, suspect this compiler before the device;
 #                    the fix is a pinned glslang build passed as
 #                    -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
 #   spirv-headers    find_package(SPIRV-Headers CONFIG REQUIRED); the distro
 #                    package ships /usr/share/cmake/SPIRV-Headers/.
 RUN apt-get update -qq \
    && apt-get install -y -qq --no-install-recommends \
        ca-certificates cmake g++ git make \
        libvulkan-dev glslc spirv-headers \
    && rm -rf /var/lib/apt/lists/*
 # Checked out detached at a full SHA, then the recorded submodule. No branch
 # name anywhere: a branch moves, this commit does not.
 ARG QWENTTS_COMMIT=5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
 RUN git clone --no-checkout https://github.com/ServeurpersoCom/qwentts.cpp /src \
    && git -C /src checkout --detach "${QWENTTS_COMMIT}" \
    && git -C /src submodule update --init --depth 1 \
    && git -C /src rev-parse HEAD \
    && git -C /src submodule status
 WORKDIR /src
 RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
        -DGGML_CCACHE=OFF \
        -DGGML_BACKEND_DL=OFF \
        -DGGML_CPU_ALL_VARIANTS=OFF \
        -DGGML_OPENMP=ON \
        -DGGML_BLAS=OFF \
        -DGGML_CUDA=OFF \
        -DGGML_HIP=OFF \
        -DGGML_VULKAN=ON \
        -DGGML_METAL=OFF \
        -DGGML_SYCL=OFF \
        -DGGML_RPC=OFF \
    && cmake --build build -j 8
 # ---------------------------------------------------------------- runtime
 FROM ${BASE_IMAGE}
 # libgomp1: the OpenMP runtime the build above links against. curl: the unit's
 # HealthCmd. ca-certificates: so that curl can speak https if anyone ever
 # points a probe at a URL with a scheme this host terminates elsewhere.
 #
 # libvulkan1 + mesa-vulkan-drivers are what make `--device Vulkan0` mean
 # anything at runtime: libvulkan1 is the loader libggml-vulkan.so dlopens, and
 # mesa-vulkan-drivers is the RADV ICD that claims the gfx1151. Loader without an
 # ICD enumerates ZERO devices, and tts-server then dies in device lookup AFTER
 # both GGUFs have been read — the journal reads as a model fault, not a missing
 # driver. Enumerated from this base on 2026-10-19 with `--device /dev/dri
 # --group-add keep-groups` plus the host's ComfyUI udev rule: "Radeon 8060S
 # Graphics (RADV GFX1151)", API 1.4.318 (the heap figure is owned by
 # [[Frank TTS Engine]]; it is the GTT-backed aperture, not the ~2 GiB DRM
 # carve-out). No NVIDIA ICD is installed on purpose: one Vulkan device in the
 # enumeration is what makes the NAME Vulkan0 safe to write down.
 RUN apt-get update -qq \
    && apt-get install -y -qq --no-install-recommends \
        ca-certificates curl libgomp1 libvulkan1 mesa-vulkan-drivers \
    && rm -rf /var/lib/apt/lists/*
 # Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
 # model directory are host state mounted read-only, and one uid across the pair
 # means one thing to remember about what is world-readable.
 #
 # `useradd`, NOT `adduser`: this trimmed ubuntu:24.04 base ships no `adduser` at
 # all (verified 2026-10-18 against this digest — `adduser` is absent from the
 # base and is not pulled in by ca-certificates/curl/libgomp1 either), so an
 # `adduser` line dies at image-build time with crun's "executable file `adduser`
 # not found in $PATH". `useradd --system --uid 10000` succeeds and prints one
 # cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`); the uid is the
 # point, and it is what matches the façade. The alpine façade Containerfile keeps
 # its own busybox `adduser -D -u 10000` — different base, different tool, same uid.
 RUN useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts
 WORKDIR /app
 COPY --from=build /src/build/tts-server /src/build/*.so* /app/
 COPY dankclaw/voiceservice/tts-engine-start.sh /app/start.sh
 # The boot warmer lives INSIDE the image because the engine publishes no port:
 # the unit reaches it with `podman exec`, and the script needs the same curl,
 # base64 and /voices the engine's own namespace has. It is a sibling of
 # start.sh, not a second entrypoint — CMD stays the server.
 COPY dankclaw/voiceservice/tts-engine-warm.sh /app/warm-voices.sh
 RUN chmod 755 /app/start.sh /app/warm-voices.sh
 # The binary's RPATH points inside the build tree; this is what makes it find
 # the libggml*.so copied beside it.
 ENV LD_LIBRARY_PATH=/app
 USER tts
 EXPOSE 8880
 CMD ["/app/start.sh"]
@@ -1,424 +0,0 @@
 #!/usr/bin/env python3
 """frank-tts — the fleet's OpenAI-compatible speech façade (component: the shim).
 One local endpoint that answers ``voice=<name>`` with audio in a cloned voice.
 Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
 ``POST /v1/audio/speech`` shape and never learn a file path: a voice is a
 directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
 the name real. A name that has no reference is not a voice — the shim answers
 400 with the list that is, instead of silently narrating in a fallback voice
 (silent fallback is how a wrong-voice endpoint stays wrong for weeks). A request
 that names NO voice at all takes ``$DEFAULT_VOICE``: that is a spoken choice in
 the env file rather than a surprise in the code, and the journal marks it
 ``(default)`` so a name nobody asked for is never mistaken for one that was.
 Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
 ``omni-http``
    Forward to an OpenAI-shaped upstream TTS server (``$TTS_UPSTREAM``) and
    carry the cloned voice with the request as ``reference_audio`` (base64 of
    ``reference.wav``). The upstream does the inference; this process only
    resolves the voice and, when asked, re-encodes to mp3 with ``ffmpeg``.
 ``llama-tts-cli``
    Run a local binary per request (``$LLAMA_TTS_BIN``), hand it the
    reference with ``--tts-speaker-file``, read the WAV it writes. ``nice -n
    19`` because this box also serves text and a voice request must never
    starve it.
 Everything comes from the environment, read at REQUEST time (no
 import-time-only globals): the same module can be imported with an injected
 ``Config`` or run with ``python3 app.py`` and the unit's env file.
 Runtime law: single process, standard library only, no build step. Audio
 bytes never reach the log — request lines and error text only.
 """
 from __future__ import annotations
 import argparse
 import base64
 import json
 import os
 import re
 import subprocess
 import sys
 import tempfile
 import urllib.error
 import urllib.parse
 import urllib.request
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 from pathlib import Path
 WAV_CONTENT_TYPE = "audio/wav"
 MP3_CONTENT_TYPE = "audio/mpeg"
 # A voice name is a plain directory name. Slashes and `..` are refused here so
 # a client-supplied name can never escape $VOICES_DIR.
 VOICE_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
 class ShimError(Exception):
    """A failure with the HTTP status the client should see."""
    def __init__(self, status, message):
        super().__init__(message)
        self.status = status
        self.message = message
 class Config:
    """Every knob of the shim. Constructed from the environment per request by
    default, or injected into ``create_server`` for tests and embedding."""
    def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
                 llama_model, llama_mmproj, llama_lang, llama_threads, port,
                 max_chars, timeout, default_voice=""):
        self.voices_dir = Path(voices_dir)
        self.backend = backend
        self.upstream = upstream
        self.upstream_model = upstream_model
        self.llama_bin = llama_bin
        self.llama_model = llama_model
        self.llama_mmproj = llama_mmproj
        self.llama_lang = llama_lang
        self.llama_threads = llama_threads
        self.port = port
        self.max_chars = max_chars
        self.timeout = timeout
        self.default_voice = default_voice
    @classmethod
    def from_env(cls, environ=None):
        env = os.environ if environ is None else environ
        return cls(
            voices_dir=env.get("VOICES_DIR", "/voices"),
            backend=(env.get("TTS_BACKEND") or "omni-http").strip().lower(),
            upstream=(env.get("TTS_UPSTREAM") or "http://host.containers.internal:8880").rstrip("/"),
            upstream_model=env.get("TTS_UPSTREAM_MODEL") or "voxcpm2",
            llama_bin=env.get("LLAMA_TTS_BIN") or "",
            llama_model=env.get("LLAMA_TTS_MODEL") or "",
            llama_mmproj=env.get("LLAMA_TTS_MMPROJ") or "",
            llama_lang=env.get("LLAMA_TTS_LANG") or "english",
            llama_threads=_int(env, "LLAMA_TTS_THREADS", 8),
            port=_int(env, "SHIM_PORT", 7772),
            max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
            timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
            default_voice=(env.get("DEFAULT_VOICE") or "").strip(),
        )
 def _int(env, name, default):
    try:
        return int(str(env.get(name, "")).strip())
    except (TypeError, ValueError):
        return default
 def list_voices(voices_dir):
    """Voice names in ``voices_dir``: directories that hold a ``reference.wav``.
    A missing or unreadable directory is an empty fleet, not an error — the
    health endpoint must still answer so the unit reports its state.
    """
    try:
        entries = sorted(p.name for p in voices_dir.iterdir()
                         if p.is_dir() and (p / "reference.wav").is_file())
    except OSError:
        return []
    return entries
 def resolve_voice(voices_dir, voice):
    """``$VOICES_DIR/<voice>/reference.wav`` for a valid name, else ``None``."""
    if not isinstance(voice, str) or not VOICE_NAME_RE.fullmatch(voice):
        return None
    reference = voices_dir / voice / "reference.wav"
    return reference if reference.is_file() else None
 def to_mp3(wav_bytes, timeout):
    """Re-encode WAV bytes to mp3 with the image's ``ffmpeg`` (pipes, no temp
    file: the audio never lands on disk to be left behind)."""
    argv = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0",
            "-f", "mp3", "pipe:1"]
    try:
        proc = subprocess.run(argv, input=wav_bytes, capture_output=True,
                              timeout=timeout)
    except FileNotFoundError:
        raise ShimError(500, "response_format=mp3 needs the ffmpeg binary, which is not on PATH")
    except subprocess.TimeoutExpired:
        raise ShimError(502, "ffmpeg mp3 conversion timed out after %gs" % timeout)
    if proc.returncode != 0 or not proc.stdout:
        raise ShimError(502, "ffmpeg mp3 conversion failed (exit %s): %s"
                        % (proc.returncode, _tail(proc.stderr)))
    return proc.stdout
 def _tail(raw):
    """Last line of a subprocess' stderr, for an error message. Text only —
    audio bytes are never echoed, and never logged."""
    text = (raw or b"").decode("utf-8", "replace").strip()
    return text.splitlines()[-1][:300] if text else "no output"
 def _error(message, voices=None):
    body = {"error": {"message": message, "type": "invalid_request_error"}}
    if voices is not None:
        body["error"]["available_voices"] = voices
    return body
 class SpeechHandler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"
    server_version = "frank-tts"
    sys_version = ""
    # -- plumbing ----------------------------------------------------------
    def config(self):
        return getattr(self.server, "config", None) or Config.from_env()
    def log_message(self, fmt, *args):
        # One prefix for the journal. Request lines and outcomes only: a
        # synthesized text, a base64 reference voice and the audio bytes
        # itself never reach stderr, in success or in failure.
        sys.stderr.write("frank-tts: %s - %s\n"
                         % (self.address_string(), fmt % args))
    def log_error(self, fmt, *args):
        # The base class routes refused requests through a separate sink; keep
        # them on the same line format so the journal reads uniformly.
        self.log_message(fmt, *args)
    def _json(self, status, obj, head=False):
        self._bytes(status, json.dumps(obj).encode("utf-8"), "application/json", head)
    def _bytes(self, status, data, content_type, head=False):
        self.send_response(status)
        self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        if not head:
            self.wfile.write(data)
    def _drain(self):
        """Read and discard a body this route will not answer. HTTP/1.1
        keep-alive makes an unread body the NEXT request line, so a client
        that reuses the connection would see a parse error it never sent."""
        if getattr(self, "_body_read", False):
            return  # already consumed; a second read would block the response
        try:
            length = int(self.headers.get("Content-Length") or 0)
        except ValueError:
            return
        if length > 0:
            self.rfile.read(length)
    def _body(self):
        """Parse the JSON request body. Never returned to a log or an error."""
        try:
            length = int(self.headers.get("Content-Length") or 0)
        except ValueError:
            raise ValueError("Content-Length is not a number")
        raw = self.rfile.read(length) if length > 0 else b""
        self._body_read = True
        if not raw:
            raise ValueError("expected a JSON body")
        parsed = json.loads(raw.decode("utf-8"))
        if not isinstance(parsed, dict):
            raise ValueError("expected a JSON object")
        return parsed
    # -- routes ------------------------------------------------------------
    def do_GET(self, head=False):
        # HEAD shares the route table: `wget --spider` (the fleet's probe
        # shape) asks for HEAD, and a 501 there would read as a dead service.
        path = urllib.parse.urlparse(self.path).path
        cfg = self.config()
        if path == "/health":
            self._json(200, {"status": "ok", "backend": cfg.backend,
                             "voices": list_voices(cfg.voices_dir)}, head)
        elif path == "/v1/audio/voices":
            self._json(200, list_voices(cfg.voices_dir), head)
        else:
            self._json(404, _error("no such endpoint: %s" % path), head)
    def do_HEAD(self):
        self.do_GET(head=True)
    def do_POST(self):
        path = urllib.parse.urlparse(self.path).path
        if path != "/v1/audio/speech":
            self._drain()
            self._json(404, _error("no such endpoint: %s" % path))
            return
        cfg = self.config()
        try:
            body = self._body()
        except (ValueError, UnicodeDecodeError) as exc:
            self._drain()
            self._json(400, _error("malformed request body: %s" % exc))
            return
        text = body.get("input")
        if not isinstance(text, str) or not text.strip():
            self._json(400, _error("`input` must be a non-empty string"))
            return
        if len(text) > cfg.max_chars:
            self._json(414, _error("input is %d characters, over the %d limit "
                                   "(SHIM_MAX_CHARS); split it or raise the limit"
                                   % (len(text), cfg.max_chars)))
            return
        fmt = str(body.get("response_format") or "wav").strip().lower()
        if fmt not in ("wav", "mp3"):
            self._json(400, _error("unsupported response_format %r; this "
                                   "endpoint serves wav and mp3" % fmt))
            return
        # `voice_id` is the alias the omp harness forwards for `voice`. A request
        # that names nothing takes the declared default — the harness has no
        # setting that feeds a voice, so voice-less calls are the normal shape
        # there, not a client bug.
        requested = body.get("voice") or body.get("voice_id")
        defaulted = not requested and bool(cfg.default_voice)
        voice = requested or cfg.default_voice
        reference = resolve_voice(cfg.voices_dir, voice)
        if reference is None:
            available = list_voices(cfg.voices_dir)
            listed = ", ".join(available) if available else "(none: no voice pack with a reference.wav in %s)" % cfg.voices_dir
            self._json(400, _error("unknown voice %r; available voices: %s"
                                   % (voice, listed), available))
            return
        model = str(body.get("model") or cfg.upstream_model)
        try:
            if cfg.backend == "omni-http":
                audio = self._speech_http(cfg, model, text, str(voice), reference)
            elif cfg.backend == "llama-tts-cli":
                audio = self._speech_cli(cfg, text, reference)
            else:
                raise ShimError(500, "unknown TTS_BACKEND %r (expected omni-http or llama-tts-cli)" % cfg.backend)
            if fmt == "mp3":
                audio = to_mp3(audio, cfg.timeout)
        except ShimError as exc:
            self._json(exc.status, _error(exc.message))
            return
        content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
        self._bytes(200, audio, content_type)
        self.log_message("speech voice=%s%s format=%s chars=%d -> %d bytes %s",
                         voice, " (default)" if defaulted else "", fmt,
                         len(text), len(audio), content_type)
    # -- backends ----------------------------------------------------------
    def _speech_http(self, cfg, model, text, voice, reference):
        """Forward to the upstream with the cloned voice riding along as
        base64 ``reference_audio``. The upstream is asked for wav always; mp3
        is our own re-encode on the way out."""
        try:
            payload = {
                "model": model,
                "input": text,
                "voice": voice,
                "response_format": "wav",
                "reference_audio": base64.b64encode(reference.read_bytes()).decode("ascii"),
            }
        except OSError as exc:
            raise ShimError(500, "cannot read %s: %s" % (reference, exc))
        url = cfg.upstream + "/v1/audio/speech"
        request = urllib.request.Request(
            url, data=json.dumps(payload).encode("utf-8"),
            headers={"Content-Type": "application/json"}, method="POST")
        try:
            with urllib.request.urlopen(request, timeout=cfg.timeout) as response:
                audio = response.read()
        except urllib.error.HTTPError as exc:
            detail = exc.read(512).decode("utf-8", "replace").strip()
            raise ShimError(502, "upstream %s returned HTTP %d: %s"
                            % (url, exc.code, detail or "no detail"))
        except urllib.error.URLError as exc:
            raise ShimError(502, "upstream %s unreachable: %s" % (url, exc.reason))
        except (TimeoutError, OSError) as exc:
            raise ShimError(502, "upstream %s failed: %s" % (url, exc))
        if not audio:
            raise ShimError(502, "upstream %s returned no audio" % url)
        return audio
    def _speech_cli(self, cfg, text, reference):
        """Run the local TTS binary once, in a temp dir that is always removed
        — also on failure, or a crash-looping binary fills /tmp."""
        if not cfg.llama_bin or not cfg.llama_model:
            raise ShimError(500, "TTS_BACKEND=llama-tts-cli needs LLAMA_TTS_BIN and LLAMA_TTS_MODEL")
        work = tempfile.TemporaryDirectory(prefix="frank-tts-")
        try:
            out_path = Path(work.name) / "speech.wav"
            argv = ["nice", "-n", "19", "env", "OMP_NUM_THREADS=%d" % cfg.llama_threads,
                    cfg.llama_bin, "-m", cfg.llama_model]
            if cfg.llama_mmproj:
                argv += ["-mm", cfg.llama_mmproj]
            argv += ["--tts-lang", cfg.llama_lang,
                     "--tts-speaker-file", str(reference),
                     "-p", text,
                     "--output", str(out_path)]
            try:
                proc = subprocess.run(argv, cwd=work.name, capture_output=True,
                                      timeout=cfg.timeout)
            except FileNotFoundError as exc:
                raise ShimError(500, "cannot run the TTS command: %s" % exc)
            except subprocess.TimeoutExpired:
                raise ShimError(502, "TTS binary timed out after %gs" % cfg.timeout)
            if proc.returncode != 0:
                raise ShimError(502, "TTS binary failed (exit %s): %s"
                                % (proc.returncode, _tail(proc.stderr)))
            if not out_path.is_file():
                raise ShimError(502, "TTS binary exited 0 but wrote no %s" % out_path.name)
            return out_path.read_bytes()
        finally:
            work.cleanup()
 class SpeechServer(ThreadingHTTPServer):
    """Threads so a health probe answers while a long synthesis runs; daemon
    threads so SIGTERM is never held hostage by a stuck request."""
    daemon_threads = True
    allow_reuse_address = True
    config = None
 def create_server(config=None, host="0.0.0.0", port=None):
    """Bind the shim. ``config``/``port`` are the injectable seams: tests pass
    a Config with a temp VOICES_DIR and ``port=0`` for an ephemeral port."""
    cfg = Config.from_env() if config is None else config
    server = SpeechServer((host, cfg.port if port is None else port), SpeechHandler)
    server.config = cfg
    return server
 def main(argv=None):
    parser = argparse.ArgumentParser(
        prog="frank-tts", description="OpenAI-compatible speech endpoint for cloned voice packs")
    parser.add_argument("--host", default="0.0.0.0",
                        help="bind address (default: all interfaces)")
    parser.add_argument("--port", type=int, default=None,
                        help="bind port (default: $SHIM_PORT, 0 = ephemeral)")
    args = parser.parse_args(argv)
    server = create_server(host=args.host, port=args.port)
    cfg = server.config
    bound = server.server_address[1]
    print("frank-tts: backend=%s voices_dir=%s listening on http://%s:%d"
          % (cfg.backend, cfg.voices_dir, args.host, bound), flush=True)
    try:
        server.serve_forever()
    except KeyboardInterrupt:
        pass
    finally:
        server.server_close()
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -1,134 +0,0 @@
 # Quadlet unit: frank-tts-engine (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.container
 # Unit name: frank-tts-engine.service (container name: frank-tts-engine)
 #
 # WHAT IT IS. The arm A TTS engine: qwentts.cpp (pinned commit
 # 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4) built into an image by
 # dankclaw/voiceservice/Containerfile.engine, running its resident `tts-server`
 # with the 1.7 B base talker Q4_K_M and the Q8_0 12 Hz tokenizer loaded ONCE at
 # startup. It is deliberately NOT a fleet endpoint: the endpoint consumers
 # address is **frank-tts on 7772** ([[Frank TTS]]), which resolves voice names,
 # re-encodes to mp3 and owns the timeout budget. Runbook: [[Frank TTS Engine]].
 #
 # WHY THIS ENGINE (Frank's call, 2026-10-18, criterion "whatever is better for
 # live output"): server-side time-to-first-audio 94-112 ms (86-451 ms end to
 # end through the façade), streaming that stays ahead of real time at warm RTF
 # 0.55-0.67, and a run-to-run spread of +-0.01 — against the mainline CLI path's
 # 0.93-0.99 at +-0.25, which crosses 1.0 often enough to stall a live stream.
 # The A/B that decided it is "Decision: arm A for live output" in
 # [[Local TTS and Voice Cloning]].
 #
 # IMAGE. Built locally by deploy-remote.sh (build_compose_image), like
 # loki/alloy/frank-tts. `Pull=never` plus the `localhost/` prefix are required:
 # this host's quadlet path cannot resolve a locally built tag from local storage
 # otherwise. The build is a network operation (it clones the pinned commit) and
 # is skipped when the Containerfile hash label already matches.
 #
 # NETWORK, AND WHY IT PUBLISHES NOTHING. The container sits on exactly one
 # network: the private `frank-tts-engine.network` (10.89.5.0/24). The façade
 # joins the same network and dials `http://frank-tts-engine:8880` — the
 # container DNS name, on the network that owns it. There is no `PublishPort` of
 # any kind, not even `127.0.0.1:8880`, and the reason is the lesson that 502'd
 # hermes: netavark DNATs published ports only, so a host-loopback publish is not
 # a path to a peer container — the façade could never have used it, and a bridge
 # container cannot reach the host's own loopback bind either (see
 # [[Container Network Namespaces]]). A publish would therefore buy exactly one
 # thing: an extra reachable surface on an engine that has no authentication, on
 # a box whose CPU is also carrying the text endpoints. Diagnostics do not need
 # it either — `podman exec frank-tts-engine curl http://127.0.0.1:8880/health`
 # speaks to the server from inside its own namespace, which is where the address
 # is valid.
 #
 # MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs:
 # the Vulkan backend copies both tensors into device memory at load, so they
 # stop counting in the process's RSS (measured after the 2026-10-19 cutover —
 # `podman stats` / `ps -o rss`). No memory limit is set: the box has the room
 # and a limit would convert a busy moment into an OOM kill mid-utterance.
 #
 # `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan
 # ICD enumerates nothing and the server dies on device lookup after it has
 # already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the
 # thing that works here: the units podman spawns carry no supplementary groups
 # at all, and this host grants render access a different way — the udev rule
 # that makes the render nodes world-writable (same rule ComfyUI rides, see
 # dankclaw/comfyui). Measured: `id` inside this container lists only the
 # container's own group, and `tts-server` still holds renderD129.
 # The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and
 # renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a
 # re-probe can invalidate, while the directory hands the container every render
 # node and the driver picks by name.
 #
 # /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
 # the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
 # container a second way onto the GPU is surface for nothing.
 #
 # THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
 # becomes known to the engine only through POST /v1/audio/voices, and the
 # registration is MEMORY ONLY — it does not survive a restart, which is exactly
 # why the warm hook below runs on every start. That hook runs INSIDE this
 # container (see ExecStartPost), so it reads the packs at /voices; `tts-server`
 # itself never opens the directory, it receives the reference audio as base64 in
 # the registration body. The façade mounts the same host directory read-only for
 # its own name list, so one directory stays the single source of a voice name.
 #
 # REGISTRY LAW, THE REASON EXECSTARTPOST EXISTS. The voice registry is memory
 # only: a restart forgets every voice, and re-registering does not warm one —
 # only synthesizing does — so a boot without this hook leaves the façade
 # answering 400 for every name. The per-voice cold penalty is real but small, and
 # not the trial's 6.2-8.0 s: measured on the built container 2026-10-18 against an
 # empty registry, the first synthesis of a freshly registered voice cost 1,883 ms
 # against 1,226-1,283 ms for the requests after it. The warm pass runs serially
 # (two short syntheses per voice, ~5 s each) and `TimeoutStartSec=900` keeps a growing library from
 # tripping systemd's default 90 s start deadline. The leading `-` on
 # ExecStartPost is load-bearing: with `Restart=always`, a warmer that exits
 # non-zero (a pack missing its reference.wav, the engine still loading past the
 # warmer's own patience) would restart the whole container and crash-loop the
 # endpoint forever. A failed warm must be a journal line, not a restart.
 #
 # The hook is `podman exec`, not a host script, because there is no address the
 # host could dial: this container publishes nothing (above). Exec-ing into it
 # also means the warmer inherits this unit's EnvironmentFile, so every knob it
 # has lives in scripts/frank-tts-engine.env beside the engine's own, and it sees
 # the same /voices and 127.0.0.1:8880 the engine does.
 #
 # HEALTH. `GET /health` answers `{"status":"ok"}` from the HTTP thread, not the
 # synthesis queue, so a busy engine still reads healthy and a dead one does not.
 # HealthStartPeriod covers the first model load. The command is argument-only
 # and needs /bin/sh — the generator wraps HealthCmd as CMD-SHELL — so the image
 # installs curl (see the opposite case in dankclaw/loki, whose distroless base
 # has no shell).
 [Unit]
 Description=frank-tts-engine (arm A qwentts.cpp TTS engine, internal :8880)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=frank-tts-engine
 Image=localhost/franksdank/frank-tts-engine:latest
 Pull=never
 Network=frank-tts-engine.network
 # No PublishPort, at all: the only reachable address this server has is the
 # bridge address the façade resolves by name. Diagnostics use `podman exec`.
 EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
 Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
 Volume=/home/dank/voices:/voices:ro
 # Render access for the iGPU (see MEMORY / THE RENDER NODE above).
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 HealthCmd=curl -fsS http://127.0.0.1:8880/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=120s
 [Service]
 Restart=always
 RestartSec=10
 # Model load (tens of seconds) plus two short syntheses per registered voice.
 TimeoutStartSec=900
 ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
 [Install]
 WantedBy=default.target
@@ -1,52 +0,0 @@
 # Quadlet network: the private bridge of the voice pair (rootless).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.network
 # Unit name: frank-tts-engine-network.service (podman network name:
 # systemd-frank-tts-engine — quadlet prefixes "systemd-", so probe the runtime
 # name, never the filename; see [[Container Network Namespaces]]).
 #
 # WHY A PRIVATE BRIDGE FOR A PAUSE THAT LOOKS PUBLIC-FACING. frank-tts (:7772,
 # published, the one endpoint consumers address) and frank-tts-engine (:8880,
 # never published, the arm A qwentts.cpp server behind it) could both have
 # stayed on the fleet-wide dankclaw_default and resolved each other by name.
 # They are split off anyway, for two reasons:
 #
 # 1. THE HOST-LOOPBACK TRAP. The phase-1 shape in `scripts/frank-tts.env` still
 #    names `host.containers.internal:8880` as the upstream, on the assumption the
 #    engine runs on the host and publishes `127.0.0.1:8880`. That cannot work:
 #    netavark DNATs published ports only, a published port is not on any podman
 #    bridge, and a container cannot reach the host's own loopback bind — the same
 #    shape that 502'd hermes, and the reason the signal-relay hop needs a host LAN
 #    bind instead ([[Container Network Namespaces]], [[Homepage Status Probes]]).
 #    Putting both containers on one committed network removes the guess: the
 #    façade dials the engine's container DNS name on the network that owns it, and
 #    the engine publishes nothing at all.
 #
 # 2. THE ENGINE HAS NO AUTH AND NO QUEUE LIMIT WORTH THE NAME. `tts-server`
 #    answers anyone who can open a socket, and at `--max-batch 1` a second
 #    caller simply waits behind the first. Left on dankclaw_default every
 #    container on that net — open-webui, hermes, the *arr proxies — could drive
 #    a CPU synthesiser that occupies every other core on a box that also serves
 #    two text endpoints. On this bridge the only dialer is frank-tts: the engine
 #    publishes no port, so not even the host has an address to reach it at, and
 #    diagnostics go through `podman exec` instead.
 #
 # Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
 # per-container userspace helper left a container that accepted TCP and reset
 # every request for two days, logging nothing). A bridge has no such helper.
 #
 # Subnet registry on dankclaw (rootless):
 #   10.88.0.0/16  default "podman" bridge
 #   10.89.0.0/24  gitea (systemd-gitea)
 #   10.89.1.0/24  compose stack (dankclaw_default)
 #   10.89.2.0/24  searxng (systemd-searxng)
 #   10.89.3.0/24  open-webui (systemd-open-webui)
 #   10.89.4.0/24  open-notebook (systemd-open-notebook)
 #   10.89.5.0/24  voice pair (systemd-frank-tts-engine)  <- this file
 # Pinned so the two container IPs stay deterministic across reboots, like the
 # other pinned nets; 10.89.5.0/24 was the next unclaimed /24 in this registry.
 [Network]
 Driver=bridge
 Subnet=10.89.5.0/24
 Gateway=10.89.5.1
@@ -1,84 +0,0 @@
 # Quadlet unit: frank-tts (rootless, boot-persistent).
 #
 # Install path on dankclaw: ~/.config/containers/systemd/frank-tts.container
 # Unit name: frank-tts.service (container name: frank-tts)
 #
 # WHAT IT IS. The fleet's one OpenAI-compatible speech endpoint: consumers
 # (omp, OpenWebUI, open-notebook) POST /v1/audio/speech with voice=<name> and
 # get audio in that cloned voice. A voice is a directory under
 # /home/dank/voices holding reference.wav; the shim refuses any name without
 # one (400 + the list that does exist) rather than narrating in a fallback
 # voice nobody chose. The engine behind it is a separate decision —
 # TTS_BACKEND in scripts/frank-tts.env picks between forwarding to the
 # OpenAI-shaped engine (`frank-tts-engine`, [[Frank TTS Engine]]) and running a
 # local CLI binary per request.
 #
 # IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
 # deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
 # pinned by digest + ffmpeg, app.py copied in, running as uid 10000.
 # Pull=never and the localhost/ prefix are required — this host's quadlet
 # path cannot resolve a locally built tag from local storage otherwise.
 #
 # NETWORK. TWO attached networks, and the order is the one the open-notebook
 # pair uses: `dankclaw_default` first (the default route, and where consumers
 # live — they reach the published 7772, and open-notebook dials
 # `http://frank-tts:7772/v1` by container name), then the voice pair's private
 # bridge, `frank-tts-engine.network`, where the arm A engine lives. The shim's
 # upstream is therefore the engine's container DNS name
 # (`http://frank-tts-engine:8880`, scripts/frank-tts.env) and NOT
 # host.containers.internal: netavark DNATs published ports only, so a bridge
 # container cannot reach a host-loopback bind — the shape that 502'd hermes —
 # and the engine publishes nothing at all. That file MUST be installed beside
 # this unit (the engine's handler does it; the façade handler re-installs it for
 # `--only frank-tts`), because with a referenced Network= file missing the v1
 # generator refuses to produce this container unit at all.
 #
 # ORDERING. Wants+After on frank-tts-engine.service, not Requires: the façade
 # has a second backend (`TTS_BACKEND=llama-tts-cli`) that needs no upstream, and
 # an engine that will not start must not take this endpoint's unit down with it.
 # When both boot, this comes up after the engine, whose ExecStartPost warm pass
 # is part of its start — so the first request of the day meets a REGISTERED voice
 # (the registry is memory-only) rather than a 400, and skips the ~0.6 s penalty
 # the first synthesis of an unwarmed voice costs.
 #
 # VOICES. /home/dank/voices is host state, never repo state, and is mounted
 # READ-ONLY: this endpoint has no business writing voice packs. Because the
 # container runs as uid 10000, the packs must be world-readable (644 files,
 # 755 directories), which is how the pack builder writes them.
 #
 # NO GPU. The shim is a façade: audio conversion via ffmpeg on the CPU and,
 # for the CLI backend, whatever CPU threads LLAMA_TTS_THREADS allows, at nice
 # 19. No /dev/kfd, no /dev/dri, no keep-groups — unlike comfyui, whose unit
 # carries them deliberately.
 [Unit]
 Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
 After=network-online.target frank-tts-engine.service
 Wants=network-online.target
 Wants=frank-tts-engine.service
 [Container]
 ContainerName=frank-tts
 Image=localhost/franksdank/frank-tts:latest
 Pull=never
 Network=dankclaw_default
 Network=frank-tts-engine.network
 PublishPort=7772:7772
 EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
 Volume=/home/dank/voices:/voices:ro
 # The probe is a probe, not a voice test: /health answers 200 with the backend
 # and the resolved voice list, so a mounted-but-empty /voices still reads
 # healthy while a dead app does not. busybox wget ships in the alpine base;
 # keep the command argument-only — the podman 5.x generator splits HealthCmd
 # on spaces (see the comfyui unit for the quoting trap).
 HealthCmd=wget --spider -q http://127.0.0.1:7772/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,124 +0,0 @@
 #!/usr/bin/env bash
 # tts-engine-start.sh — the entrypoint of the frank-tts-engine image
 # (dankclaw/voiceservice/Containerfile.engine). It exists for one reason: the
 # knobs of the arm A engine are command-line flags, while the fleet's knobs
 # live in scripts/frank-tts-engine.env and reach the container through
 # EnvironmentFile=. This is the translation layer, and nothing else — no
 # supervision, no retries, no logging of its own. PID 1 stays the server, so
 # `podman stop` and the unit's Restart=always drive it directly.
 #
 # THE FLAG THAT MUST STAY ABSENT:
 #
 #   --threads / -t   does not exist in this build. src/backend.h computes the
 #                    CPU thread count as std::thread::hardware_concurrency()/2
 #                    (the comment there: GEMM shares SIMD units across
 #                    hyperthreads), and there is no flag to override it. So a
 #                    CPU affinity pin (CPUQuota=, taskset, a cpuset) silently
 #                    changes the engine's thread count instead of just its
 #                    placement — pinning is deliberately not done anywhere in
 #                    this stack. See the env file header for the same warning
 #                    aimed at the operator.
 #
 # --device IS here, but conditionally: it reaches the binary only when
 # QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
 # takes its own default device, which is CPU because the CPU backend is always
 # built. One line in scripts/frank-tts-engine.env moves the endpoint between the
 # two shapes, with no rebuild. What must never reach argv is a device NAME the
 # loader does not enumerate: that is not refused by the argument parser, it dies
 # during device lookup with the loader's own error, so the journal reads as a
 # load that failed rather than a configuration refusal.
 #
 # Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
 # the Q8_0 tokenizer, one request at a time, on the private bridge, on the
 # gfx1151 iGPU. An empty environment still boots that shape.
 set -euo pipefail
 # The binary path. Overridable for one reason: the bats seam runs a stub
 # instead of a 1.2 GiB model. Operators never set it — the image ships exactly
 # one binary at /app/tts-server.
 : "${QWENTTS_BIN:=/app/tts-server}"
 # Paths are CONTAINER paths (the host directories they come from are named in
 # scripts/frank-tts-engine.env and mounted by the Quadlet unit).
 : "${QWENTTS_MODEL:=/models/qwen-talker-1.7b-base-Q4_K_M.gguf}"
 : "${QWENTTS_CODEC:=/models/qwen-tokenizer-12hz-Q8_0.gguf}"
 # Reported by GET /v1/models, and what scripts/frank-tts.env's
 # TTS_UPSTREAM_MODEL names. A model id, not a filename: the file name would
 # leak a quantization choice into every consumer's config.
 : "${QWENTTS_ALIAS:=frank-tts-engine}"
 # 0.0.0.0 inside the container is not "public". tts-server defaults to
 # 127.0.0.1 (its own usage prints `--host <ip> Listen address (default:
 # 127.0.0.1)`), and inside a container netns a loopback bind is reachable by
 # nothing outside that namespace — the façade dials this container by its bridge
 # address on the private voice network, which is why the address has to be
 # broad. The unit PUBLISHES NOTHING (see
 # dankclaw/voiceservice/quadlet/frank-tts-engine.container: no PublishPort at
 # all, not even 127.0.0.1:8880), so the only namespaces that can reach
 # 0.0.0.0:8880 are that bridge and this container's own loopback — and its own
 # loopback is exactly where the boot warmer and the health probe speak.
 : "${QWENTTS_HOST:=0.0.0.0}"
 : "${QWENTTS_PORT:=8880}"
 # The language label used when a request omits one. `auto` is the upstream
 # default and what the trial ran.
 : "${QWENTTS_LANG:=auto}"
 # One request at a time. The server-side queue is a serialised FIFO at this
 # setting, which is what keeps the per-voice timings predictable. A second
 # concurrent synthesis shares the device with the first — and on this box that
 # device is also carrying frank-dragoon — so it buys no throughput and it
 # doubles the wait for the caller behind it. The engine's own default is also 1;
 # written here because it is a decision, not an accident.
 : "${QWENTTS_MAX_BATCH:=1}"
 # The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
 # the gfx1151 iGPU on this host. An index would follow whatever order the
 # loader's device list returns; a name says which device this endpoint means.
 # The container sees exactly one Vulkan device because the image carries the
 # Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
 #
 # EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
 # the binary picks its own default, and the CPU backend is always built — which
 # is why moving this endpoint between the iGPU and the CPU is one line in
 # scripts/frank-tts-engine.env and not a rebuild.
 #
 # That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
 # fires on a set-but-empty variable as well as an unset one, so the rollback line
 # `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
 # on the shared GPU with no way out short of a rebuild. Only UNSET takes the
 # default.
 if [[ ! -v QWENTTS_DEVICE ]]; then
    QWENTTS_DEVICE=Vulkan0
 fi
 # Fail loudly BEFORE binding, so the journal says which artifact is missing
 # instead of the unit reporting a container that starts and dies. This is the
 # self-healing path: Restart=always re-runs it every RestartSec, and the same
 # sentence repeats until the operator fetches the weights
 # (scripts/download-frank-tts-engine-qwentts.sh).
 for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
    if [[ ! -r "$artifact" ]]; then
        echo "frank-tts-engine: cannot read ${artifact}; run scripts/download-frank-tts-engine-qwentts.sh on the host (the /models mount is read-only and holds the two GGUF files)" >&2
        exit 1
    fi
 done
 ARGS=(
    --model "$QWENTTS_MODEL"
    --codec "$QWENTTS_CODEC"
    --alias "$QWENTTS_ALIAS"
    --host "$QWENTTS_HOST"
    --port "$QWENTTS_PORT"
    --lang "$QWENTTS_LANG"
    --max-batch "$QWENTTS_MAX_BATCH"
 )
 # Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
 # because "the rollback shape" is argv identical to the pre-iGPU endpoint.
 if [[ -n "$QWENTTS_DEVICE" ]]; then
    ARGS+=(--device "$QWENTTS_DEVICE")
 fi
 exec "$QWENTTS_BIN" "${ARGS[@]}"
@@ -1,222 +0,0 @@
 #!/usr/bin/env bash
 # tts-engine-warm.sh — the boot warmer of the arm A TTS engine, shipped inside
 # the frank-tts-engine image at /app/warm-voices.sh
 # (dankclaw/voiceservice/Containerfile.engine).
 #
 # WHO RUNS IT. Two callers, same binary:
 #
 #   1. The Quadlet unit, as
 #      `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
 #      — after every start of the container, including every boot.
 #      `podman exec` rather than a host script because the engine publishes no
 #      port at all, so there is no address the host could dial; and because an
 #      exec inherits the unit's EnvironmentFile, which is what puts every knob of
 #      this script in scripts/frank-tts-engine.env next to the engine's own.
 #   2. scripts/deploy-remote.sh's deploy_frank_tts_engine, once per deploy, so a
 #      voice that did not warm becomes a deploy result instead of only a journal
 #      line (the unit's `-` prefix deliberately cannot fail the unit).
 #
 # WHY IT EXISTS AT ALL (the registry law). The voice registry is MEMORY ONLY: a
 # restart forgets every voice, and re-registering a voice does not warm it — only
 # synthesizing does. A boot that registers nothing therefore leaves frank-tts
 # answering 400 for every name, so this script registers every pack it finds. The
 # per-voice cold penalty is real but small, and smaller than the trial reported:
 # measured on the deployed build 2026-10-18 against an empty registry, the FIRST
 # synthesis of a freshly registered voice cost 1,883 ms against 1,226-1,283 ms for
 # the requests after it — about 0.6 s, not the trial's 6.2-8.0 s. Each voice gets
 # two identical requests so the journal carries both numbers, `voice <name>
 # cold=…ms warm=…ms`; cold ≈ warm is the expected shape, and a large gap is what
 # would mean the snapshot path is broken.
 #
 # THE VOICES. /voices — the host directory /home/dank/voices mounted read-only by
 # the unit, the same packs frank-tts mounts for its own name list. A pack is a
 # directory that contains reference.wav; everything else in it (manifest.json, the
 # engine's reference.spk / reference.rvq extraction, a 16 kHz variant) is not read
 # here. The registration payload is `{"name": <dir>, "wav_b64": <base64 of
 # reference.wav>}` with NO ref_text, which selects the embedding-only (x-vector)
 # voice — the shipped default, warm RTF ~0.57-0.63 against ICL's 0.67, and the one
 # this fleet can produce honestly because it has no trustworthy transcript for the
 # audition crops. Adding ref_text (with the codes) would flip that voice to
 # in-context-learning mode: better fidelity on a good transcript, slower, and only
 # worth it for batch work like a podcast render. That is a per-voice opt-in this
 # script deliberately does NOT exercise, and the door stays open: a pack that
 # wants ICL needs its text in the pack plus one key in the payload below.
 #
 # A lone `spk_b64` is a 400 (src/tts-server.h:480 — it is `wav_b64` or BOTH
 # `spk_b64` and `rvq_b64`), which is why the WAV travels and the .spk stays home.
 #
 # NOTHING HERE IS A SECRET, and nothing is logged that could be: the audio bytes
 # are discarded, the synthesized text comes from the env file, and the reference
 # clip is base64'd into a request body that is written to a temporary file and
 # handed to curl as `@file` — never echoed, and never placed on a command line,
 # because of the size law explained at the mktemp below.
 set -euo pipefail
 # The engine's own address, from inside its namespace. Loopback is valid HERE:
 # this script runs in the container, next to the server. There is deliberately no
 # host-visible address to use instead — the unit publishes nothing, because a
 # host-loopback publish would be unreachable from the façade anyway (netavark
 # DNATs published ports only) while adding a surface on an unauthenticated
 # server.
 : "${ENGINE_URL:=http://127.0.0.1:8880}"
 # Container path; the host directory behind it is named in the Quadlet unit.
 : "${VOICES_DIR:=/voices}"
 # How long to wait for the engine to answer /health. Model load dominates: the
 # two GGUFs (1.2 GiB + 291 MB) are read and the context built before the HTTP
 # thread binds, and a boot races every other unit on this box.
 : "${HEALTH_WAIT:=300}"
 # One short sentence, long enough to exercise the codec snapshot and short enough
 # that warming a library of voices is seconds, not minutes. Never varied per run:
 # the cold/warm numbers in the journal should mean the same thing every boot. The
 # synthesis asks for `wav`, the buffered path — the streamed path (`pcm`) is
 # different audio at the same seed, and `wav` is also what frank-tts asks for.
 : "${WARM_TEXT:=Warm up, one two three.}"
 # Per-request ceiling. A cold voice is 6-8 s by measurement; anything approaching
 # this is a stalled engine and should be reported rather than waited on forever —
 # systemd counts the whole warm pass against the unit's start deadline.
 : "${REQUEST_TIMEOUT:=120}"
 # The warm text goes into a JSON string literal by interpolation. Two steps make
 # whatever the two readers of scripts/frank-tts-engine.env deliver arrive as the
 # sentence itself:
 #
 # 1. Unwrap ONE shell-style quoting layer. bash (`set -a; source`) strips the
 #    quotes around a value with spaces; podman's `--env-file` does NOT — measured
 #    2026-10-18 against the image's pinned base, `WARM_TEXT="…"` reaches the
 #    container WITH its quotes. Unwrapped only when the whole value is exactly one
 #    quoted string (starts AND ends with `"` and holds no quote inside), so a
 #    sentence that legitimately contains quotes is never mangled.
 # 2. Escape the two characters that would corrupt or end a JSON string: a
 #    backslash, and a double quote.
 #
 # Without step 1 the body is `{"input":""Warm up…"",…}`, the engine's JSON parser
 # rejects it, every voice stays cold, and the boot still reads healthy. Voice NAMES
 # need neither step: the character class checked below admits no quote, no
 # backslash and no control byte.
 warm_text="$WARM_TEXT"
 if (( ${#warm_text} >= 2 )) && [[ "$warm_text" == \"*\" ]]; then
    inner="${warm_text:1:${#warm_text}-2}"
    [[ "$inner" != *'"'* ]] && warm_text="$inner"
 fi
 warm_json="${warm_text//\\/\\\\}"
 warm_json="${warm_json//\"/\\\"}"
 # The registration body is a FILE, never an argument. The kernel caps a single
 # command-line string at 32 pages — MAX_ARG_STRLEN = 131,072 bytes — and a shipped
 # reference clip base64's to about 1.27 MB, so passing the body as one argument
 # fails with E2BIG before curl is even executed. The first real boot of the unit
 # (2026-10-18, the three shipped packs) proved it end to end: `/usr/bin/curl:
 # Argument list too long` for every voice, reported as the misleading
 # "registration failed (HTTP no response)". `mktemp` lands in the container's
 # /tmp as the unprivileged `tts` user; one file for the whole pass, overwritten
 # per voice, removed on any exit — it is container scratch, not host state.
 body_file="$(mktemp -t frank-tts-warm.XXXXXX)"
 trap 'rm -f "$body_file"' EXIT
 fail=0
 log() { printf 'frank-tts-engine-warm: %s\n' "$*"; }
 # Wait for /health. It answers from the HTTP thread, so a 200 means the server is
 # listening; it says nothing about voices, which is what the rest of this script
 # is for.
 ready=0
 for ((i = 1; i <= HEALTH_WAIT; i++)); do
    if curl -fsS --max-time 5 "${ENGINE_URL}/health" >/dev/null 2>&1; then
        ready=1
        break
    fi
    sleep 1
 done
 if (( ready == 0 )); then
    log "engine did not answer ${ENGINE_URL}/health within ${HEALTH_WAIT}s; nothing warmed"
    exit 1
 fi
 if [[ ! -d "$VOICES_DIR" ]]; then
    log "no voice directory at ${VOICES_DIR}; nothing to warm (frank-tts will answer 400 with an empty voice list until packs are built)"
    exit 0
 fi
 warmed=0
 for dir in "$VOICES_DIR"/*/; do
    [[ -d "$dir" ]] || continue
    name="${dir%/}"
    name="${name##*/}"
    if [[ ! -f "${dir}reference.wav" ]]; then
        continue          # not a pack; frank-tts refuses this name too
    fi
    # The engine lowercases every voice name (tts_voice_name) while frank-tts
    # matches the directory name exactly, so a mixed-case pack would work by
    # accident on one side and not the other. Refuse it loudly instead. Pack
    # names are also JSON string content here, so the character class is what
    # keeps a directory name from writing structure into the request body.
    if [[ ! "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
        log "voice '${name}': name must be lowercase [a-z0-9._-] (the engine lowercases names; the shim does not) — skipped"
        fail=1
        continue
    fi
    # The audio enters through a redirection rather than a command substitution:
    # the file is several hundred KB, and both a shell variable and an argument of
    # that size are what this script used to die on.
    {
        printf '{"name":"%s","wav_b64":"' "$name"
        base64 -w0 "${dir}reference.wav"
        printf '"}'
    } > "$body_file"
    if ! code="$(curl -sS --max-time "$REQUEST_TIMEOUT" -o /dev/null -w '%{http_code}' \
            -H 'Content-Type: application/json' --data-binary "@${body_file}" \
            "${ENGINE_URL}/v1/audio/voices")" || [[ "$code" != "200" ]]; then
        log "voice '${name}': registration failed (HTTP ${code:-no response}) — the shim will answer 400 for this name"
        fail=1
        continue
    fi
    # The cold pass. Its output is discarded; the point is the codec snapshot it
    # leaves inside the engine.
    start=$(date +%s%3N)
    if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
            -H 'Content-Type: application/json' \
            --data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
            "${ENGINE_URL}/v1/audio/speech"; then
        log "voice '${name}': registered, but the warm synthesis failed — first live request stays cold"
        fail=1
        continue
    fi
    cold=$(( $(date +%s%3N) - start ))
    # The warm pass, same request. This is the number the fleet actually
    # experiences, and it is what "the second request answers under a second" is
    # measured against.
    start=$(date +%s%3N)
    if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
            -H 'Content-Type: application/json' \
            --data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
            "${ENGINE_URL}/v1/audio/speech"; then
        log "voice '${name}': warm synthesis failed after a cold one that worked"
        fail=1
        continue
    fi
    warm=$(( $(date +%s%3N) - start ))
    log "voice ${name} cold=${cold}ms warm=${warm}ms"
    warmed=$(( warmed + 1 ))
 done
 # Failure first, emptiness second, and not the other way round: a pass where
 # every voice failed also has `warmed == 0`, and reading that as "nothing to do,
 # success" is how an endpoint that answers 400 for every voice name gets deployed
 # on a Friday.
 if (( fail )); then
    log "${warmed} voice(s) warm, at least one failed (see the lines above)"
    exit 1
 fi
 if (( warmed == 0 )); then
    log "no voice with a reference.wav under ${VOICES_DIR}; engine is up but speaks nothing"
    exit 0
 fi
 log "${warmed} voice(s) registered and warm"
@@ -1,156 +0,0 @@
 #!/usr/bin/env python3
 """HTTP front door for the frank-medium single-GPU replicas.
 Why two replicas instead of TP=2: one vLLM process spanning both R9700s needs
 RCCL collectives, and RCCL's kernels require PCIe atomics on every
 participating GPU. The second card sits behind chipset bridges advertising
 `AtomicOpsCap: Routing-`, which no software can enable (see the runbook), so
 no collective can launch on it. Two single-GPU replicas use both cards; this
 service keeps the fleet contract of one endpoint on :7770, streaming included.
 Routing decisions live in routing.py — spread single-turn requests, pin
 continuations to the replica whose prefix cache holds their context. This file
 only moves bytes: parse enough of the request to route it, forward it verbatim,
 stream the answer back, and report replica health on /health.
 """
 from __future__ import annotations
 import asyncio
 import os
 from contextlib import suppress
 from aiohttp import ClientSession, ClientTimeout, web
 import routing
 PORT = int(os.environ.get("PORT", "7770"))
 HEALTH_TTL = float(os.environ.get("HEALTH_TTL", "10"))
 # Hop-by-hop headers must not be forwarded (RFC 9110 §7.6.1) — forwarding
 # transfer-encoding or content-length of a rewritten body breaks clients.
 HOP_BY_HOP = {
    "host", "content-length", "transfer-encoding", "connection",
    "keep-alive", "proxy-authenticate", "proxy-authorization", "te",
    "trailers", "upgrade",
 }
 UPSTREAMS = routing.UPSTREAMS
 # Replica health, refreshed by the watcher. Module state rather than aiohttp app
 # state: the application is frozen once the site is up.
 _UP: list[bool] = [True] * len(UPSTREAMS)
 async def health(session: ClientSession) -> web.Response:
    results = await asyncio.gather(
        *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
        return_exceptions=True,
    )
    lines, up = [], 0
    for url, res in zip(UPSTREAMS, results):
        if isinstance(res, Exception) or res.status != 200:
            lines.append(f"{url} DOWN")
        else:
            up += 1
            lines.append(f"{url} UP")
    return web.Response(status=200 if up else 503, text="\n".join(lines) + "\n")
 async def watch_health() -> None:
    """Refresh the replica up/down table so routing never targets a dead one.
    One HTTP request per replica per interval, in exchange for never handing a
    client a 502 after a restart.
    """
    global _UP
    session = ClientSession()
    try:
        while True:
            res = await asyncio.gather(
                *(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
                return_exceptions=True,
            )
            _UP = [not isinstance(r, Exception) and r.status == 200 for r in res]
            await asyncio.sleep(HEALTH_TTL)
    finally:
        await session.close()
 async def proxy(request: web.Request) -> web.StreamResponse:
    session: ClientSession = request.app["session"]
    if request.path == "/health":
        return await health(session)
    body = await request.read()
    headers = {k: v for k, v in request.headers.items() if k.lower() not in HOP_BY_HOP}
    # Fail over before anything is written to the client. A replica that
    # restarts is invisible to the health watcher for up to HEALTH_TTL, and a
    # deploy restarts both replicas — without this retry a 10 s window of
    # requests would 502 on a service that has a live card the whole time.
    # vLLM's OpenAI endpoints are stateless per request, so re-sending a
    # request that never reached an engine is safe; a request that failed
    # mid-stream is not retried (that path is below).
    tried: set[int] = set()
    upstream = None
    last_error = "no healthy frank-medium replica"
    while len(tried) < len(UPSTREAMS):
        idx = routing.route(body, request.remote, _UP)
        if idx < 0 or idx in tried:
            break
        tried.add(idx)
        target = UPSTREAMS[idx] + request.path_qs
        try:
            upstream = await session.request(
                request.method, target, headers=headers, data=body or None,
                timeout=ClientTimeout(total=None, sock_connect=10, sock_read=3600),
            )
            break
        except Exception as exc:
            # Demote at once: the watcher will confirm, but every request until
            # then should skip this replica rather than pay the same timeout.
            # Logged because a silent failover hides a card that is down.
            _UP[idx] = False
            last_error = f"{target} unreachable: {exc}"
            print(f"proxy: failover — {target} unreachable ({exc}); retrying "
                  f"another replica", flush=True)
    if upstream is None:
        # 503 when no replica was even reachable through the routing table,
        # 502 when replicas were tried and all of them failed.
        status = 502 if tried else 503
        return web.Response(status=status, text=f"proxy: {last_error}\n")
    out = web.StreamResponse(status=upstream.status, reason=upstream.reason)
    for k, v in upstream.headers.items():
        if k.lower() not in HOP_BY_HOP:
            out.headers[k] = v
    await out.prepare(request)
    try:
        async for chunk in upstream.content.iter_any():
            await out.write(chunk)
    except Exception:
        # The client went away mid-stream (an aborted generation). Nothing to
        # report: the engine sees its own cancellation.
        pass
    finally:
        upstream.release()
    with suppress(Exception):
        await out.write_eof()
    return out
 async def main() -> None:
    app = web.Application(client_max_size=1024 ** 3)
    app["session"] = ClientSession()
    app.router.add_route("*", "/{tail:.*}", proxy)
    runner = web.AppRunner(app)
    await runner.setup()
    await web.TCPSite(runner, "0.0.0.0", PORT).start()
    print(f"proxy: :{PORT} -> {', '.join(UPSTREAMS)} routing={routing.ROUTING} "
          f"key={routing.AFFINITY_BYTES}B known_max={routing.KNOWN_MAX}", flush=True)
    watcher = asyncio.create_task(watch_health())
    with suppress(asyncio.CancelledError):
        await asyncio.gather(watcher)
 if __name__ == "__main__":
    with suppress(KeyboardInterrupt):
        asyncio.run(main())
@@ -1,98 +0,0 @@
 # Quadlet unit: frank-medium replica A (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/frank-medium-a.container
 # Unit name: frank-medium-a.service (container name: frank-medium-a)
 #
 # WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
 # tensor-parallel process across both R9700s (see frank-medium.container, kept
 # as the target). That shape cannot run on this host: RCCL's collective kernels
 # carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
 # dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
 # hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
 # second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
 # `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
 # atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
 # therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
 # The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
 # atomics. Result: every collective that touches GPU 2 dies at launch.
 # Two independent single-GPU replicas use both cards instead; the proxy unit
 # keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
 #
 # SHAPE. vLLM binds 0.0.0.0:7771 on the host network directly; no PublishPort.
 # MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
 # whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
 # 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
 # this shape instead of using the 2x7551-32624 pin, which is only valid at
 # TP=2/262144.
 #
 # DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=0 (the first R9700). The
 # 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
 # Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
 # first and HIP indexes into the filtered list, so a single-valued pair
 # (ROCR=1,HIP=1) leaves an empty device set.
 #
 # DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
 # HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
 [Unit]
 Description=frank-medium replica A — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #1
 After=network-online.target
 [Container]
 ContainerName=frank-medium-a
 Image=localhost/radiance-vllm:0.9.3
 AddDevice=/dev/kfd
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 # The upstream launcher's exact podman posture (rootless).
 PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
 Environment=HIP_VISIBLE_DEVICES=0
 Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
 Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
 Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
 Environment=SERVED_NAMES=frank-medium
 Environment=PORT=7771
 Environment=TP=1
 Environment=MAXSEQS=4
 Environment=CHUNK=8192
 Environment=MAXLEN=114688
 Environment=GPU_UTIL=0.98
 Environment=KV_MEM=0
 Environment=SPEC_METHOD=dflash
 Environment=SPEC=7
 # Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
 Volume=/home/dank/models:/models
 # Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
 Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
 # Compile cache. Shared with the other replica and with the TP=2 target: the
 # entries are content-addressed, so shapes simply add subdirectories.
 Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
 # The pinned radiance clone: patches applied at every container start.
 Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
 # The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
 # shipped r4d.so predates the GDN overflow fix and NaNs this model.
 Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
 # Entrypoint + healthcheck from this checkout, mounted read-only.
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
 # The entrypoint must be the container command; the image's own entrypoint
 # (which the patched script execs at the end) must not run unpatched.
 PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
 # 200 only once the engine serves; during first-start compilation the
 # container is legitimately unhealthy. HealthCmd is split on spaces into an
 # argv array by the podman 5.x v1 generator, hence the file-based probe.
 HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
 HealthInterval=60s
 HealthTimeout=30s
 HealthRetries=3
 HealthStartPeriod=1800s
 Pull=never
 [Service]
 # First start compiles Triton/inductor kernels (several minutes) before the
 # engine loads ~20 GiB of weights.
 TimeoutStartSec=1800
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,98 +0,0 @@
 # Quadlet unit: frank-medium replica B (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/frank-medium-b.container
 # Unit name: frank-medium-b.service (container name: frank-medium-a)
 #
 # WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
 # tensor-parallel process across both R9700s (see frank-medium.container, kept
 # as the target). That shape cannot run on this host: RCCL's collective kernels
 # carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
 # dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
 # hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
 # second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
 # `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
 # atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
 # therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
 # The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
 # atomics. Result: every collective that touches GPU 2 dies at launch.
 # Two independent single-GPU replicas use both cards instead; the proxy unit
 # keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
 #
 # SHAPE. vLLM binds 0.0.0.0:7772 on the host network directly; no PublishPort.
 # MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
 # whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
 # 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
 # this shape instead of using the 2x7551-32624 pin, which is only valid at
 # TP=2/262144.
 #
 # DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=1 (the second R9700). The
 # 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
 # Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
 # first and HIP indexes into the filtered list, so a single-valued pair
 # (ROCR=1,HIP=1) leaves an empty device set.
 #
 # DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
 # HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
 [Unit]
 Description=frank-medium replica B — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #2
 After=network-online.target
 [Container]
 ContainerName=frank-medium-b
 Image=localhost/radiance-vllm:0.9.3
 AddDevice=/dev/kfd
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 # The upstream launcher's exact podman posture (rootless).
 PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
 Environment=HIP_VISIBLE_DEVICES=1
 Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
 Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
 Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
 Environment=SERVED_NAMES=frank-medium
 Environment=PORT=7772
 Environment=TP=1
 Environment=MAXSEQS=4
 Environment=CHUNK=8192
 Environment=MAXLEN=114688
 Environment=GPU_UTIL=0.98
 Environment=KV_MEM=0
 Environment=SPEC_METHOD=dflash
 Environment=SPEC=7
 # Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
 Volume=/home/dank/models:/models
 # Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
 Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
 # Compile cache. Shared with the other replica and with the TP=2 target: the
 # entries are content-addressed, so shapes simply add subdirectories.
 Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
 # The pinned radiance clone: patches applied at every container start.
 Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
 # The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
 # shipped r4d.so predates the GDN overflow fix and NaNs this model.
 Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
 # Entrypoint + healthcheck from this checkout, mounted read-only.
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
 # The entrypoint must be the container command; the image's own entrypoint
 # (which the patched script execs at the end) must not run unpatched.
 PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
 # 200 only once the engine serves; during first-start compilation the
 # container is legitimately unhealthy. HealthCmd is split on spaces into an
 # argv array by the podman 5.x v1 generator, hence the file-based probe.
 HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
 HealthInterval=60s
 HealthTimeout=30s
 HealthRetries=3
 HealthStartPeriod=1800s
 Pull=never
 [Service]
 # First start compiles Triton/inductor kernels (several minutes) before the
 # engine loads ~20 GiB of weights.
 TimeoutStartSec=1800
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,61 +0,0 @@
 # Quadlet unit: frank-medium proxy (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/frank-medium-proxy.container
 # Unit name: frank-medium-proxy.service (container name: frank-medium-proxy)
 #
 # WHAT IT IS. A prefix-affinity reverse proxy in front of the two single-GPU
 # frank-medium replicas (:7771 on R9700 #1, :7772 on R9700 #2). It preserves
 # the fleet contract: one endpoint on host port 7770 answering as the served
 # model name frank-medium, OpenAI /v1, streaming included.
 #
 # WHY IT EXISTS. TP=2 — one vLLM process spanning both cards — is impossible
 # on this host (see frank-medium-a.container and the runbook: RCCL's kernels
 # need PCIe atomics, and the second card's chipset path advertises
 # AtomicOpsCap: Routing-). Two replicas are how both cards get used.
 #
 # ROUTING. Requests are keyed on a hash of their leading 4 KiB of prompt text,
 # so a conversation family — an agent loop re-sending the same system prompt,
 # tool list and file context every turn — keeps landing on the same replica,
 # where --enable-prefix-caching has that prefix warm. Plain round-robin threw
 # half of those cache hits away. A replica that is down is skipped, and
 # ROUTING=roundrobin restores the old behaviour for A/B measurement. /health
 # answers 200 while at least one replica is up and names the ones that are
 # down. The proxy runs on host networking so it can bind 7770 and reach the
 # replicas on loopback.
 #
 # The script is a few dozen lines of aiohttp, mounted read-only out of
 # /home/dank/serverAppConfigs (deploy.sh pulls that checkout before dispatch).
 # It reuses the same image as the replicas: aiohttp is already there and no
 # extra image has to be pulled on this host.
 [Unit]
 Description=frank-medium proxy — prefix-affinity in front of the two single-GPU replicas
 After=network-online.target
 Wants=frank-medium-a.service frank-medium-b.service
 After=frank-medium-a.service frank-medium-b.service
 [Container]
 ContainerName=frank-medium-proxy
 Image=localhost/radiance-vllm:0.9.3
 PodmanArgs=--network=host
 Environment=PORT=7770
 Environment=UPSTREAMS=http://127.0.0.1:7771,http://127.0.0.1:7772
 # The proxy is two files: routing.py holds the replica-selection policy, the
 # server holds the HTTP plumbing. Both are mounted because the server imports
 # the policy by name from its own directory (/usr/local/bin is sys.path[0]).
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/routing.py:/usr/local/bin/routing.py:ro
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/affinity-proxy.py:/usr/local/bin/affinity-proxy.py:ro
 Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
 PodmanArgs=--entrypoint /usr/local/bin/affinity-proxy.py
 HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
 HealthInterval=60s
 HealthTimeout=30s
 HealthRetries=3
 HealthStartPeriod=60s
 Pull=never
 [Service]
 Restart=on-failure
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,129 +0,0 @@
 #!/usr/bin/env python3
 """Routing policy for the frank-medium proxy: which replica serves a request.
 Kept apart from the HTTP plumbing in affinity-proxy.py so the decisions can be
 tested without aiohttp, a network or a GPU — the first version of these rules
 shipped with a bug that only appeared on a continuation request, and a test
 that loads this module catches that class of mistake directly.
 Two cards, two behaviours, and the workload decides which is right:
 * A **single-turn** request is one member of a batch (a benchmark run, a burst
  of parallel subagents). Spreading those over both replicas lets both cards
  prefill at once; measured on this host, a 16-request burst of 15k-token
  prompts finishes in 18.6 s spread versus 30.1 s pinned.
 * A **continuation** re-sends a prefix the replica has already prefilled. Pinned
  to that replica the turn costs a cache lookup; spread, half the turns pay a
  cold prefill again (measured: 1096 ms versus 2676 ms mean warm-turn TTFT on
  90k-token contexts).
 ROUTING picks the strategy; the default `adaptive` applies both rules above.
 `affinity` and `roundrobin` exist so the choice can be re-measured, and are not
 production settings.
 """
 from __future__ import annotations
 import collections
 import hashlib
 import itertools
 import json
 import os
 UPSTREAMS = [u.strip() for u in os.environ.get(
    "UPSTREAMS", "http://127.0.0.1:7771,http://127.0.0.1:7772").split(",") if u.strip()]
 ROUTING = os.environ.get("ROUTING", "adaptive").strip().lower()
 # Leading bytes of prompt text that identify a conversation's shared prefix.
 # 4 KiB covers a system prompt plus the start of the first user turn, which is
 # where agent loops keep the text they resend every turn.
 AFFINITY_BYTES = int(os.environ.get("AFFINITY_BYTES", "4096"))
 # Prefixes remembered for pinning. Bounded so a long-lived proxy cannot grow
 # without limit; 4096 conversations is far beyond what this fleet runs at once.
 KNOWN_MAX = int(os.environ.get("KNOWN_MAX", "4096"))
 _counter = itertools.count()
 # digest -> replica index, insertion-ordered for O(1) LRU eviction. A prefix is
 # remembered as soon as any request carries it; only a continuation is pinned
 # there, because only a continuation can profit from the cache entry.
 _known: "collections.OrderedDict[bytes, int]" = collections.OrderedDict()
 def affinity_key(body: bytes, remote: str | None) -> tuple[bytes, bool]:
    """Key a request by its shared prefix, and report whether it continues one.
    Chat and completions bodies both carry their prompt text in JSON, so the
    leading bytes of that text are the prefix; a body we cannot parse (or an
    endpoint with no prompt, e.g. /v1/models) falls back to the client address,
    which still beats nothing for a single-client loop.
    """
    fallback = (remote or "anonymous").encode()
    try:
        payload = json.loads(body)
    except Exception:
        return fallback, False
    text = ""
    continuation = False
    if isinstance(payload, dict):
        if isinstance(payload.get("prompt"), str):
            text = payload["prompt"]
        elif isinstance(payload.get("messages"), list):
            continuation = len(payload["messages"]) > 1
            parts = []
            for msg in payload["messages"]:
                if isinstance(msg, dict):
                    content = msg.get("content")
                    if isinstance(content, str):
                        parts.append(content)
                    elif isinstance(content, list):  # multimodal content blocks
                        parts.extend(
                            blk.get("text", "") for blk in content
                            if isinstance(blk, dict) and isinstance(blk.get("text"), str)
                        )
            text = "\n".join(parts)
    if not text:
        return fallback, continuation
    return text[:AFFINITY_BYTES].encode("utf-8", "replace"), continuation
 def pick(key: bytes, continuation: bool, up: list[bool]) -> int:
    """Replica for a request, or -1 when none is available.
    Down replicas are never selected — a rollout must not send traffic to an
    engine that is still compiling its graphs — and a pin whose replica is down
    is re-established on a live one rather than kept pointing at a corpse.
    """
    digest = hashlib.sha1(key).digest()
    if ROUTING == "roundrobin":
        start = next(_counter) % len(UPSTREAMS)
    elif ROUTING == "adaptive":
        entry = _known.get(digest)
        if continuation and entry is not None and up[entry]:
            _known.move_to_end(digest)
            return entry
        # Anything that is not a continuation spreads by counter — including a
        # batch of requests that happen to share one prefix. Hashing the prefix
        # would send every member of such a batch to whichever card the hash
        # names (measured: 29.8 s for a 16-request burst, against 18.6 s when
        # spread). The counter is only advanced by these requests, so batch
        # members alternate across the cards.
        start = next(_counter) % len(UPSTREAMS)
    else:  # affinity
        start = int.from_bytes(digest[:8], "big") % len(UPSTREAMS)
    for offset in range(len(UPSTREAMS)):
        idx = (start + offset) % len(UPSTREAMS)
        if up[idx]:
            if ROUTING == "adaptive":
                _known[digest] = idx
                _known.move_to_end(digest)
                if len(_known) > KNOWN_MAX:
                    _known.popitem(last=False)
            return idx
    return -1
 def route(body: bytes, remote: str | None, up: list[bool]) -> int:
    """Convenience wrapper: decode the request body and choose a replica."""
    return pick(*affinity_key(body, remote), up)
@@ -1,22 +0,0 @@
 # Dankserver stack secrets and configuration.
 # Copy to .env and fill real values. Never commit real secrets.
 # RomM admin credentials (first boot)
 ROMM_USERNAME=changeme
 ROMM_PASSWORD=changeme
 ROMM_AUTH_SECRET_KEY=changeme
 # RomM database (MariaDB)
 ROMM_DB_USER=romm
 ROMM_DB_PASSWD=changeme
 MARIADB_ROOT_PASSWORD=changeme
 # RomM Redis
 # (no password needed for local Redis)
 # Copyparty
 COPAY_PASS=changeme
 # Jellyfin
 JELLYFIN_PUBLISH_SERVER_URL=http://192.168.0.235:8096
@@ -1,63 +0,0 @@
 # Quadlet unit: Calibre Content Server (rootless, boot-persistent).
 #
 # Install path on dankserver:
 # ~/.config/containers/systemd/calibre-server.container
 # Unit name: calibre-server.service (podman 5.x drops the container- prefix)
 #
 # The image is the CURRENT linuxserver/calibre (9.15.0), which is a desktop
 # image (Selkies web-streamed GUI on 8080/8181) with the headless
 # calibre-server available inside. The compose-era snippet in
 # serverAppConfigs-docs/Dankserver/Calibre.md predates the desktop revamp;
 # the dedicated headless images (ta264/docker-calibre, agentgreen) are
 # dormant (calibre 6.10 / 2017), so this unit runs calibre-server as the
 # image's s6-overlay main service via Exec=; the desktop also runs but its
 # ports are internal-only (see the port map below).
 #
 # Port map: Network=pasta is port-FORWARD mode on this host (the container
 # keeps its own netns; only PublishPort'd ports reach the host — verified
 # 2026-09-21: 8080/8090/8182 inside the container are not visible on the
 # host, only the 8084 forward). So the ONLY exposed surface is:
 #   8090 -> host 8084  calibre-server (the content server; 8080 inside the
 #                      container is the image's own nginx, 8083 on the host
 #                      is calibre-web)
 # The desktop GUI (nginx 8080/8181 + WS 8082, inside the container) runs but
 # is internal-only: no PublishPort. Exposing it is one PublishPort line away
 # if the browser GUI is ever wanted (the basic-auth credential is already in
 # the env file).
 # Auth: calibre-server 9.x keeps its users in a SQLite userdb (the pw column
 # is PLAINTEXT, so the file is mode 600). The single admin account is
 # bootstrapped once by the deploy handler (deploy_calibre_server) via
 # `calibre-server --manage-users -- add`, password piped on stdin with
 # printf (a trailing newline from echo lands in the stored password and
 # breaks every login). The desktop's basic auth (CUSTOM_USER/CUSTOM_PASSWORD)
 # reuses the same credential pair. Both live in the machine-local env file;
 # this unit holds no secrets.
 #
 # --auth-mode=basic is required: on a non-SSL port the server defaults to
 # Digest auth, which rejects plain Basic clients with 400 "Unsupported
 # authentication method".
 [Unit]
 Description=Calibre content server container
 After=network.target
 [Container]
 ContainerName=calibre-server
 Image=lscr.io/linuxserver/calibre:9.15.0
 Network=pasta
 EnvironmentFile=/home/dank/Containers/calibre-server/env
 Environment=PUID=1000
 Environment=PGID=1000
 Environment=TZ=America/Vancouver
 Volume=/home/dank/Containers/calibre-web/books:/books
 Volume=/home/dank/Containers/calibre-server/userdb.sqlite:/userdb.sqlite
 Exec=calibre-server --port 8090 --enable-auth --auth-mode=basic --userdb /userdb.sqlite /books
 PublishPort=8084:8090
 [Service]
 TimeoutStartSec=300
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,43 +0,0 @@
 # Quadlet unit: Calibre-Web library server (rootless, boot-persistent).
 #
 # Install path on dankserver:
 # ~/.config/containers/systemd/calibre-web.container
 # Unit name: calibre-web.service (podman 5.x drops the container- prefix)
 #
 # The data roots already lived under /home/dank/Containers/calibre-web in the
 # compose era; this unit keeps those exact paths. The image pin is the exact
 # tag of the image the live container runs (image id 986277563e76), so the
 # cutover recreation changes zero bits of application code. No secrets: this
 # unit takes no EnvironmentFile.
 [Unit]
 Description=Calibre-Web library server container
 After=network.target
 [Container]
 ContainerName=calibre-web
 Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402
 Network=pasta
 Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
 Environment=LANG=en_US.UTF-8
 Environment=LANGUAGE=en_US.UTF-8
 Environment=LSIO_FIRST_PARTY=true
 Environment=OAUTHLIB_RELAX_TOKEN_SCOPE=1
 Environment=PGID=1000
 Environment=PUID=1000
 Environment=QTWEBENGINE_CHROMIUM_FLAGS=--no-sandbox
 Environment=S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
 Environment=S6_STAGE2_HOOK=/docker-mods
 Environment=TERM=xterm
 Environment=TZ=America/Vancouver
 Environment=VIRTUAL_ENV=/lsiopy
 PublishPort=8083:8083
 Volume=/home/dank/Containers/calibre-web/config:/config
 Volume=/home/dank/Containers/calibre-web/books:/books
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,154 +0,0 @@
 # Dankserver stack — media & library services.
 # Deploy: cd /home/dank/serverAppConfigs && podman-compose -f dankserver/compose.yaml up -d
 # See ../docs/Dankserver/ for service documentation.
 networks:
  appnet:
    driver: bridge
 services:
  # ═══════════════════════════════════════════════
  # RomM — retro game manager (requires MariaDB + Redis)
  # ═══════════════════════════════════════════════
  mariadb:
    image: docker.io/mariadb:11
    container_name: romm-mariadb
    restart: unless-stopped
    environment:
      MARIADB_ROOT_PASSWORD: "${MARIADB_ROOT_PASSWORD}"
      MARIADB_DATABASE: romm
      MARIADB_USER: "${ROMM_DB_USER}"
      MARIADB_PASSWORD: "${ROMM_DB_PASSWD}"
    volumes:
      - /opt/romm/mariadb:/var/lib/mysql
    healthcheck:
      test: ["CMD", "mariadb-admin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 3
  redis:
    image: docker.io/redis:7-alpine
    container_name: romm-redis
    restart: unless-stopped
    volumes:
      - /opt/romm/redis:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3
  romm:
    image: ghcr.io/rommapp/romm:latest
    container_name: romm
    restart: unless-stopped
    environment:
      ROMM_USERNAME: "${ROMM_USERNAME}"
      ROMM_PASSWORD: "${ROMM_PASSWORD}"
      ROMM_AUTH_SECRET_KEY: "${ROMM_AUTH_SECRET_KEY}"
      DB_HOST: mariadb
      DB_NAME: romm
      DB_USER: "${ROMM_DB_USER}"
      DB_PASSWD: "${ROMM_DB_PASSWD}"
      REDIS_HOST: redis
      HASHEOUS_API_ENABLED: "true"
    ports:
      - "8085:8080"
    volumes:
      - /opt/romm/library:/romm/library
      - /opt/romm/resources:/romm/resources
      - /opt/romm/config:/romm/config
    depends_on:
      mariadb:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
  # ═══════════════════════════════════════════════
  # Copyparty — file sharing and streaming
  # ═══════════════════════════════════════════════
  copyparty:
    image: ghcr.io/9001/copyparty-ac:latest
    container_name: copyparty
    restart: unless-stopped
    # read-only since 2026-08-31: copyparty perms are per-field "letters[,user]"
    # with no user = every unauthenticated request. Anonymous gets r (browse +
    # download; the guarded files.franksdank.cloud chain reaches copyparty
    # unauthenticated after authentik). dank gets ra (admin panel). No w/m/d
    # anywhere: uploads, moves, deletes rejected server-side. Interface: `--help-accounts` in-image.
    command: ["-a", "dank:${COPAY_PASS}", "-v", "/w::r:ra,dank"]
    ports:
      - "3923:3923"
    volumes:
      - /mnt/entropy:/w:ro
      - /opt/copyparty/cfg:/cfg
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:3923/')"]
      interval: 30s
      timeout: 10s
      retries: 3
  # ═══════════════════════════════════════════════
  # Jellyfin — media server (movies, TV, music)
  # ═══════════════════════════════════════════════
  jellyfin:
    image: jellyfin/jellyfin:latest
    container_name: jellyfin
    restart: unless-stopped
    ports:
      - "8096:8096/tcp"
      - "8920:8920/tcp"
      - "7359:7359/udp"
      - "1900:1900/udp"
    environment:
      - JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
      - DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
    volumes:
      - /opt/jellyfin/config:/config
      - /opt/jellyfin/cache:/cache
      - /mnt/entropy:/data:ro
    devices:
      - /dev/dri:/dev/dri
    group_add:
      - "992"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
      interval: 30s
      timeout: 10s
      retries: 3
  # ═══════════════════════════════════════════════
  # Calibre-Web — e-book web frontend
  # ═══════════════════════════════════════════════
  calibre-web:
    image: lscr.io/linuxserver/calibre-web:0.6.27-ls402
    container_name: calibre-web
    restart: unless-stopped
    environment:
      PUID: "1000"
      PGID: "1000"
      TZ: Etc/UTC
      OAUTHLIB_RELAX_TOKEN_SCOPE: "1"
    ports:
      - "8083:8083"
    volumes:
      - /opt/calibre-web/config:/config
      - /opt/calibre-web/books:/books
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8083/"]
      interval: 30s
      timeout: 10s
      retries: 3
  # node-exporter left this file on 2026-09-16 for
  # dankserver/observability/quadlet/node-exporter.container: a compose-only
  # container does not survive a reboot here — it died `Exited (143)` and stayed
  # dead, leaving Prometheus' node-dankserver job scraping nothing — and the
  # compose form bound the wildcard address. One owner per container name.
@@ -1,38 +0,0 @@
 # Quadlet unit: Copyparty file server (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/copyparty.container
 # Unit name: copyparty.service (podman 5.x drops the container- prefix)
 #
 # The image's default entrypoint is /bin/ash /z/cpp.sh -c /z/initcfg; the
 # Command= line below reproduces the compose-era argv exactly: the admin
 # account is passed as -a <user>:<password>. ${COPAY_ACCOUNT} and
 # ${COPAY_PASSWORD} are expanded by systemd at ExecStart time from the
 # host-local env file /home/dank/Containers/copyparty/env (mode 0600).
 # This file holds no secrets. The deployer refuses to start this unit before
 # that env file exists, so a missing env file fails the unit instead of
 # creating an admin account with a garbage password.
 #
 # /w is the entropy volume, mounted read-only: nothing writes into /mnt/entropy
 # anymore; every consumer of the share is a read-only client.
 [Unit]
 Description=Copyparty file server container
 After=network.target
 [Container]
 ContainerName=copyparty
 Image=ghcr.io/9001/copyparty-ac:latest
 Network=dankserver_appnet
 EnvironmentFile=/home/dank/Containers/copyparty/env
 Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD} -v /w::r:ra,dank
 Volume=/mnt/entropy:/w:ro
 Volume=/home/dank/Containers/copyparty/cfg:/cfg
 PublishPort=3923:3923
 [Service]
 EnvironmentFile=/home/dank/Containers/copyparty/env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,57 +0,0 @@
 # Quadlet unit: Jellyfin media server (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/jellyfin.container
 # Unit name: jellyfin.service (podman 5.x drops the container- prefix)
 #
 # Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
 # cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
 # anywhere writes to the entropy volume. Every
 # creation-time environment variable of the compose-era container is kept as
 # an Environment= line (zero drift); none of them are secrets, so this unit
 # needs no EnvironmentFile.
 [Unit]
 Description=Jellyfin media server container
 After=network.target
 [Container]
 ContainerName=jellyfin
 Image=ghcr.io/jellyfin/jellyfin:latest
 Network=dankserver_appnet
 Environment=DEBIAN_FRONTEND=noninteractive
 Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
 Environment=JELLYFIN_CACHE_DIR=/cache
 Environment=JELLYFIN_CONFIG_DIR=/config/config
 Environment=JELLYFIN_DATA_DIR=/config
 Environment=JELLYFIN_FFMPEG=/usr/lib/jellyfin-ffmpeg/ffmpeg
 Environment=JELLYFIN_LOG_DIR=/config/log
 Environment=JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
 Environment=JELLYFIN_WEB_DIR=/jellyfin/jellyfin-web
 Environment=LANG=en_US.UTF-8
 Environment=LANGUAGE=en_US:en
 Environment=LC_ALL=en_US.UTF-8
 Environment=LD_PRELOAD=/usr/lib/jellyfin/libjemalloc.so.2
 Environment=MALLOC_TRIM_THRESHOLD_=131072
 Environment=NVIDIA_DRIVER_CAPABILITIES=compute,video,utility
 Environment=NVIDIA_VISIBLE_DEVICES=all
 Environment=XDG_CACHE_HOME=/cache
 PublishPort=8096:8096
 PublishPort=8920:8920
 PublishPort=7359:7359/udp
 PublishPort=1900:1900/udp
 Volume=/home/dank/Containers/jellyfin/config:/config
 Volume=/home/dank/Containers/jellyfin/cache:/cache
 Volume=/mnt/entropy:/data:ro
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 HealthCmd=curl -f http://localhost:8096/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,58 +0,0 @@
 # Quadlet unit: node-exporter (rootless, boot-persistent) — DANKSERVER.
 #
 # Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
 # Unit name: node-exporter.service (container name: node-exporter)
 #
 # WHY THIS FILE EXISTS. Until 2026-09-16 the Dankserver node-exporter ran only
 # from `dankserver/compose.yaml` (the "Observability" block, `restart:
 # unless-stopped`). Rootless Podman does not restart bare containers after the
 # explicit stop that shutdown performs, so it died at a reboot and stayed dead:
 # `podman ps -a` showed it `Exited (143)` and Prometheus' `node-dankserver` job
 # had been scraping nothing since. The compose entry also bound the default
 # wildcard address, which is the unauthenticated-metrics-API-on-every-interface
 # posture that was already removed on Dankclaw. This unit replaces both: one
 # owner per container name (the compose service is deleted), explicit listen
 # addresses, and a generator-provided boot path.
 #
 # HOST NAMESPACE. Networking and PID in the host's namespaces: the collectors
 # report the host's own interfaces, connections, processes and cgroups. The
 # host's /proc, /sys and / are mounted read-only under /host and /rootfs, and
 # node_exporter writes nothing.
 #
 # LISTEN ADDRESSES. Exactly two:
 #   - 127.0.0.1:9100   for host-local probes
 #   - 192.168.0.235:9100 for Prometheus, which scrapes this host from Dankclaw
 #     (shared/prometheus/prometheus.yml, job "node-dankserver"). It cannot use
 #     loopback across hosts, and node_exporter has no authentication, so the
 #     wildcard bind stays off the table. This is the same deliberate LAN
 #     listener posture the observability runbook documents.
 #
 # MIRROR. Keep this file in step with dankclaw/observability/quadlet/
 # node-exporter.container; only the Description and the LAN address differ.
 [Unit]
 Description=Prometheus node exporter (Dankserver host metrics)
 After=network-online.target
 Wants=network-online.target
 [Container]
 ContainerName=node-exporter
 Image=docker.io/prom/node-exporter:latest
 Network=host
 PodmanArgs=--pid=host
 Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.235:9100
 Volume=/proc:/host/proc:ro
 Volume=/sys:/host/sys:ro
 Volume=/:/rootfs:ro
 HealthCmd=wget --spider -q http://localhost:9100/metrics
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 HealthStartPeriod=30s
 [Service]
 Restart=always
 RestartSec=10
 [Install]
 WantedBy=default.target
@@ -1,22 +0,0 @@
 [Unit]
 Description=Plex Media Server (quadlet, rootless fleet identity)
 After=network-online.target
 Wants=network-online.target
 [Container]
 Image=lscr.io/linuxserver/plex:1.43.4.10903-e5521bd8c-ls325
 ContainerName=plex
 Network=dankserver_appnet:alias=plex
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
 Volume=/var/lib/plexmediaserver:/config
 Volume=/mnt/entropy:/mnt/entropy:ro
 PublishPort=32400:32400
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,34 +0,0 @@
 # Quadlet unit: MariaDB backing store for RomM (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/romm-mariadb.container
 # Unit name: romm-mariadb.service (podman 5.x drops the container- prefix)
 #
 # The alias `mariadb` on dankserver_appnet is what makes DB_HOST=mariadb in
 # the RomM unit resolve. Secrets (MARIADB_ROOT_PASSWORD, MARIADB_PASSWORD)
 # live in the shared host-local env file /home/dank/Containers/romm/env
 # (mode 0600); this file holds no secrets.
 [Unit]
 Description=RomM MariaDB database container
 After=network.target
 [Container]
 ContainerName=romm-mariadb
 Image=docker.io/library/mariadb:11
 Network=dankserver_appnet:alias=mariadb
 EnvironmentFile=/home/dank/Containers/romm/env
 Environment=MARIADB_DATABASE=romm
 Environment=MARIADB_USER=romm
 Volume=/home/dank/Containers/romm/mariadb:/var/lib/mysql
 HealthCmd=mariadb-admin ping -h localhost
 HealthInterval=10s
 HealthTimeout=5s
 HealthRetries=3
 [Service]
 EnvironmentFile=/home/dank/Containers/romm/env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,28 +0,0 @@
 # Quadlet unit: Redis cache for RomM (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/romm-redis.container
 # Unit name: romm-redis.service (podman 5.x drops the container- prefix)
 #
 # The alias `redis` on dankserver_appnet is what makes REDIS_HOST=redis in
 # the RomM unit resolve. No secrets: this unit takes no EnvironmentFile.
 [Unit]
 Description=RomM Redis cache container
 After=network.target
 [Container]
 ContainerName=romm-redis
 Image=docker.io/library/redis:7-alpine
 Network=dankserver_appnet:alias=redis
 Volume=/home/dank/Containers/romm/redis:/data
 HealthCmd=redis-cli ping
 HealthInterval=10s
 HealthTimeout=5s
 HealthRetries=3
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,42 +0,0 @@
 # Quadlet unit: RomM library manager (rootless, boot-persistent).
 #
 # Install path on dankserver: ~/.config/containers/systemd/romm.container
 # Unit name: romm.service (podman 5.x drops the container- prefix)
 #
 # The whole data root lives at /home/dank/Containers/romm and is mounted at
 # /romm, so config/, library/, resources/ and assets/ sit inside it exactly as
 # the compose-era bind mounts + anonymous volume did. Secrets (DB_PASSWD,
 # ROMM_AUTH_SECRET_KEY, ROMM_PASSWORD, SCREENSCRAPER_DEV_*) live in the
 # host-local env file /home/dank/Containers/romm/env (mode 0600); this file
 # holds no secrets. The network dankserver_appnet must exist before first
 # start (it is the compose-era network; on a fresh host create it first).
 [Unit]
 Description=RomM library manager container
 Wants=romm-mariadb.service
 After=romm-mariadb.service romm-redis.service network.target
 [Container]
 ContainerName=romm
 Image=ghcr.io/rommapp/romm:latest
 Network=dankserver_appnet
 EnvironmentFile=/home/dank/Containers/romm/env
 Environment=DB_HOST=mariadb
 Environment=DB_NAME=romm
 Environment=DB_USER=romm
 Environment=REDIS_HOST=redis
 Environment=HASHEOUS_API_ENABLED=true
 PublishPort=8085:8080
 Volume=/home/dank/Containers/romm:/romm
 HealthCmd=curl -f http://localhost:8080/health
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 [Service]
 EnvironmentFile=/home/dank/Containers/romm/env
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -1,25 +0,0 @@
 [Unit]
 Description=Tautulli (quadlet, rootless fleet identity)
 After=network-online.target
 Wants=network-online.target
 [Container]
 Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245
 ContainerName=tautulli
 Network=dankserver_appnet:alias=tautulli
 Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
 Volume=/home/dank/Containers/tautulli:/config
 PublishPort=8181:8181
 HealthCmd=curl -fs http://localhost:8181/home
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
 [Service]
 Restart=always
 RestartSec=5
 [Install]
 WantedBy=default.target
@@ -0,0 +1,93 @@
 #!/usr/bin/env bash
 set -euo pipefail
 # deploy.sh — Deploy inference scripts to remote servers
 #
 # Usage: ./deploy.sh [TARGET] [SCRIPT]
 #   TARGET: dankserver, dankclaw, or all
 #   SCRIPT: Script name to deploy (default: all scripts in scripts/)
 TARGET="${1:-all}"
 SCRIPT="${2:-}"
 # Remote configuration
 declare -A REMOTE_HOSTS
 REMOTE_HOSTS=(
    ["dankserver"]="dank@dankserver"
    ["dankclaw"]="dank@dankclaw"
 )
 declare -A REMOTE_DIRS
 REMOTE_DIRS=(
    ["dankserver"]="/home/dank/vllm"
    ["dankclaw"]="/home/dank/vllm"
 )
 declare -A REMOTE_SERVICES
 REMOTE_SERVICES=(
    ["dankserver"]="vllm-awq-qwen36.service"
    ["dankclaw"]="vllm-thinkingcap.service"
 )
 deploy_to() {
    local target="$1"
    local remote="${REMOTE_HOSTS[$target]}"
    local remote_dir="${REMOTE_DIRS[$target]}"
    local service="${REMOTE_SERVICES[$target]}"
    echo "=== Deploying to $target ==="
    # Create remote directory if it doesn't exist
    sshpass -p 'Assface.3241' ssh "$remote" "mkdir -p $remote_dir"
    # Deploy scripts
    if [ -n "$SCRIPT" ]; then
        if [ -f "scripts/$SCRIPT" ]; then
            echo "  Deploying scripts/$SCRIPT..."
            sshpass -p 'Assface.3241' scp "scripts/$SCRIPT" "$remote:$remote_dir/$SCRIPT"
        else
            echo "  ERROR: scripts/$SCRIPT not found"
            return 1
        fi
    else
        for script in scripts/*.sh; do
            if [ -f "$script" ]; then
                local name=$(basename "$script")
                echo "  Deploying $name..."
                sshpass -p 'Assface.3241' scp "$script" "$remote:$remote_dir/$name"
            fi
        done
    fi
    # Make scripts executable
    if [ -n "$SCRIPT" ]; then
        sshpass -p 'Assface.3241' ssh "$remote" "chmod +x $remote_dir/$SCRIPT"
    else
        sshpass -p 'Assface.3241' ssh "$remote" "chmod +x $remote_dir/*.sh"
    fi
    # Restart service if it exists
    echo "  Restarting $service..."
    sshpass -p 'Assface.3241' ssh "$remote" "echo 'Assface.3241' | sudo -S systemctl restart $service 2>/dev/null || echo 'Service $service not found or failed'"
    echo "  Deploy to $target complete"
    echo ""
 }
 case "$TARGET" in
    dankserver|dankclaw)
        deploy_to "$TARGET"
        ;;
    all)
        for target in "${!REMOTE_HOSTS[@]}"; do
            deploy_to "$target"
        done
        ;;
    *)
        echo "ERROR: Unknown target $TARGET"
        echo "Usage: $0 [dankserver|dankclaw|all] [script]"
        exit 1
        ;;
 esac
 echo "=== Deployment complete ==="
@@ -0,0 +1,81 @@
 # Dankclaw System Specs
 ## Server Overview
 Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
 ## Hardware Specifications
 ### CPU
 | Property | Value |
 |----------|-------|
 | Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
 | Architecture | x86_64 |
 | Cores | 16 (1 socket, 1 thread per core) |
 | Max MHz | 5187.50 |
 | Min MHz | 625.00 |
 | NUMA Nodes | 1 (node 0, 62909 MB) |
 ### GPU (Integrated)
 | Property | Value |
 |----------|-------|
 | Model | AMD Radeon 8060S (Strix Halo) |
 | Device ID | 0x1586 |
 | Architecture | gfx1151 (Strix Halo) |
 | GPU-Visible Memory | 96 GiB (ROCm addressable range) |
 | Physical RAM | 30 GiB (shared CPU + GPU) |
 ### ROCm
 | Property | Value |
 |----------|-------|
 | ROCk Module | Loaded |
 | Runtime Version | 1.18 |
 | ROCm Version | 7.2.4 (from AMD repos) |
 | Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
 | GPU Arch | gfx1151 (Strix Halo) |
 | GPU Global Memory | 96 GiB (overcommitted, physical is 30 GiB) |
 ### Memory
 | Property | Value |
 |----------|-------|
 | System RAM | 30 GiB (shared CPU + GPU) |
 | GPU-Visible (ROCm) | 96 GiB (overcommitted addressable range) |
 | Swap | 8.0 GiB |
 | /dev/shm | ~15 GiB (half of RAM) |
 ### Storage
 | Filesystem | Size | Used | Avail | Use% |
 |------------|------|------|-------|------|
 | / (nvme0n1p2) | 1.9T | 391G | 1.4T | 22% |
 ### OS and Software
 | Property | Value |
 |----------|-------|
 | OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
 | Kernel | 7.0.0-29-generic |
 | Python | 3.14.4 (system), 3.12.13 (vLLM venv) |
 | vLLM | 0.26.0+rocm723 |
 | uv | 0.12.2 |
 ## VLLM Relevance
 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
 - ROCm reports 96 GiB GPU memory (overcommitted). Physical RAM is 30 GiB. vLLM sees 96 GiB and will try to use it. Set `--gpu-memory-utilization` conservatively (0.80-0.85) to avoid OOM when the OS needs memory.
 - Only 8 GiB swap. Set `--swap-space` conservatively (4-8 GiB).
 - ROCm 7.2.4 from AMD repos. vLLM ROCm support requires ROCm 6.3+.
 - Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Use a Python 3.12 venv.
 - Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
 - 1.4 TB free disk space is ample for model storage.
 - `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
 ## See Also
 - [[VLLM Overview]]
 - [[Shared Memory Architecture Dankclaw]]
 - [[RAM CPU Offloading in VLLM]]
 - [[Running Qwen3.6 27B FP8 on Dankclaw]]
@@ -0,0 +1,68 @@
 # Dankserver System Specs
 ## Server Overview
 Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 24.04 with an RTX 3090 and 62 GB of system RAM.
 ## Hardware Specifications
 ### CPU
 | Property | Value |
 |----------|-------|
 | Architecture | x86_64 |
 | Vendor | AuthenticAMD |
 | Cores | 8 (1 socket) |
 | NUMA Nodes | 1 (node 0, 63503 MB) |
 ### GPU
 | Property | Value |
 |----------|-------|
 | Model | NVIDIA GeForce RTX 3090 |
 | VRAM | 24576 MiB (24 GB) |
 | Driver | 595.84 |
 | CUDA Version | 13.2 |
 | CUDA Toolkit | 12.0 (nvcc) |
 ### Memory
 | Property | Value |
 |----------|-------|
 | System RAM | 62 GiB |
 | Swap | 127 GiB |
 | /dev/shm | 32 GiB |
 ### Storage
 | Filesystem | Size | Used | Avail | Use% |
 |------------|------|------|-------|------|
 | / (nvme0n1p2) | 915G | 849G | 20G | 98% |
 | /mnt/entropy (dm-0) | 91T | 86T | 977G | 99% |
 ### OS and Software
 | Property | Value |
 |----------|-------|
 | OS | Ubuntu 24.04.4 LTS (Noble Numbat) |
 | Kernel | 7.0.0-28-generic |
 | Python | 3.12.3 |
 ### Active Services
 A llama-server process occupies approximately 20 GB of the RTX 3090 VRAM.
 ## VLLM Relevance
 - 24 GB VRAM limits model size. Use FP8 quantization for large models.
 - 62 GB system RAM provides ample swap space for KV cache offloading.
 - 127 GB swap supports large `--swap-space` values for CPU KV cache paging.
 - Root disk has only 20 GB free. Store models on /mnt/entropy.
 - CUDA 12.0 and Python 3.12.3 are compatible with vLLM.
 ## See Also
 - [[VLLM Overview]]
 - [[Hybrid CPU GPU Processing Dankserver]]
 - [[RAM CPU Offloading in VLLM]]
 - [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
@@ -0,0 +1,213 @@
 # Hybrid CPU GPU Processing Dankserver
 ## Overview
 Dankserver uses NVIDIA GPU with system RAM for hybrid processing. VLLM splits model weights onto GPU and moves KV cache between GPU VRAM and system RAM. This allows larger models to run on GPUs with limited VRAM.
 ## Architecture
 ### Model Weights
 Model weights stay on GPU. VLLM loads the entire model into GPU memory. You cannot offload model weights to CPU. The GPU must hold all model parameters.
 ### KV Cache
 KV cache moves between GPU and CPU:
 1. Active KV cache pages stay in GPU VRAM
 2. Inactive pages swap to CPU RAM
 3. Swapped pages write to disk swap space if RAM is full
 4. Pages reload to GPU when needed
 ### Compute Distribution
 - GPU handles all matrix multiplication and attention computation
 - CPU handles tokenization, detokenization, and input processing
 - CPU handles KV cache swap operations
 - GPU worker processes run on CPU cores via multiprocessing
 ## Optimal Settings for Dankserver
 ### GPU Memory Utilization
 Set `--gpu-memory-utilization` based on your GPU:
 | GPU | Recommended Value |
 |-----|-------------------|
 | 24 GB (RTX 3090/4090) | 0.7-0.8 |
 | 40 GB (A100) | 0.8-0.9 |
 | 80 GB (A100 80GB) | 0.85-0.95 |
 | 80 GB (H100) | 0.9-0.95 |
 Lower values leave more GPU memory for KV cache. This reduces swap frequency.
 ### Swap Space
 Set `--swap-space` based on system RAM:
 | System RAM | Recommended Swap |
 |------------|------------------|
 | 32 GB | 8-16 GB |
 | 64 GB | 16-32 GB |
 | 128 GB | 32-64 GB |
 | 256 GB | 64-128 GB |
 More swap space allows longer contexts and more concurrent requests.
 ### Tensor Parallelism
 Use tensor parallelism for large models:
 ```
 vllm serve meta-llama/Llama-3.1-70B-Instruct \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.85 \
    --swap-space 16
 ```
 ### KV Cache Data Type
 Use FP8 KV cache for Ampere and newer GPUs:
 ```
 vllm serve Qwen/Qwen3-30B-A3B-FP8 \
    --kv-cache-dtype fp8 \
    --gpu-memory-utilization 0.9 \
    --swap-space 16
 ```
 FP8 KV cache reduces memory usage by 50% compared to BF16.
 ## NUMA Optimization
 Dankserver may have multiple NUMA nodes. Optimize with:
 ```
 vllm serve model \
    --tensor-parallel-size 4 \
    --numa-bind
 ```
 This pins each GPU worker to its nearest NUMA node. This reduces memory access latency.
 For explicit NUMA binding:
 ```
 vllm serve model \
    --tensor-parallel-size 4 \
    --numa-bind \
    --numa-bind-nodes 0 0 1 1
 ```
 ## CPU Resource Requirements
 VLLM needs CPU cores for:
 - 1 API server process
 - 1 engine core process
 - N GPU worker processes (one per GPU)
 Minimum physical cores = 2 + N (where N is GPU count).
 For Dankserver with 1 GPU:
 - Minimum 3 physical CPU cores
 - Recommended 8+ physical CPU cores
 ## Performance Tuning
 ### Chunked Prefill
 Chunked prefill is enabled by default. Tune with:
 ```
 vllm serve model \
    --max-num-batched-tokens 8192
 ```
 Higher values improve throughput. Lower values improve latency.
 ### CUDA Graphs
 CUDA graphs improve throughput. Disable for faster startup:
 ```
 vllm serve model \
    --enforce-eager
 ```
 ### Compilation Cache
 Reuse torch.compile artifacts:
 ```
 export VLLM_CACHE_ROOT=~/.cache/vllm
 vllm serve model
 ```
 Copy cache between boots for faster startup.
 ### Skip Memory Profiling
 Skip memory profiling on repeated boots:
 ```
 vllm serve model \
    --kv-cache-memory <value_from_first_boot_log>
 ```
 ## Monitoring
 Check GPU memory:
 ```
 nvidia-smi
 ```
 Check for preemption warnings in VLLM logs.
 Check CPU utilization:
 ```
 htop
 ```
 ## Example Commands
 ### Single GPU, Medium Model
 ```
 vllm serve Qwen/Qwen3-30B-A3B \
    --gpu-memory-utilization 0.85 \
    --swap-space 16 \
    --max-num-batched-tokens 4096 \
    --max-num-seqs 128
 ```
 ### Single GPU, FP8 Model
 ```
 vllm serve Qwen/Qwen3-30B-A3B-FP8 \
    --kv-cache-dtype fp8 \
    --gpu-memory-utilization 0.9 \
    --swap-space 16 \
    --max-num-batched-tokens 8192
 ```
 ### Multi-GPU, Large Model
 ```
 vllm serve meta-llama/Llama-3.1-70B-Instruct \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.9 \
    --swap-space 32 \
    --max-model-len 8192
 ```
 ## See Also
 - [[VLLM Overview]]
 - [[VLLM Index]]
 - [[RAM CPU Offloading in VLLM]]
 - [[Shared Memory Architecture Dankclaw]]
 - [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
@@ -0,0 +1,179 @@
 # RAM CPU Offloading in VLLM
 ## Overview
 VLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. VLLM does not offload model weights to CPU. Instead, VLLM moves KV cache data between GPU and CPU.
 ## How CPU Offloading Works
 VLLM manages KV cache as pages. Each page holds attention key and value tensors for a set of tokens. When GPU memory is full, VLLM can move pages to CPU memory or disk swap space.
 The KV cache swap mechanism works as follows:
 1. VLLM allocates GPU blocks for active KV cache pages
 2. When GPU blocks are exhausted, VLLM swaps inactive pages to CPU
 3. Swapped pages are stored in a CPU memory buffer
 4. When a swapped page is needed, VLLM loads it back to GPU
 5. If CPU buffer is full, pages are written to disk swap space
 ## Key Parameters
 ### gpu_memory_utilization
 This parameter controls the fraction of GPU memory used for KV cache.
 | Value | Description |
 |-------|-------------|
 | 0.7 | Uses 70% of GPU memory for KV cache |
 | 0.8 | Uses 80% of GPU memory for KV cache |
 | 0.9 | Uses 90% of GPU memory for KV cache (default) |
 | 0.95 | Uses 95% of GPU memory for KV cache |
 Lower values leave more GPU memory for model weights. Higher values allow more concurrent requests but increase swap frequency.
 ### swap_space
 This parameter sets the CPU swap space in GiB.
 | Value | Description |
 |-------|-------------|
 | 0 | No CPU swap space |
 | 4 | 4 GiB swap space (default) |
 | 8 | 8 GiB swap space |
 | 16 | 16 GiB swap space |
 | 32 | 32 GiB swap space |
 | 64 | 64 GiB swap space |
 The swap space acts as an extension of GPU memory. When GPU blocks are full, VLLM moves KV cache pages to swap space.
 ### max_model_len
 This parameter sets the maximum context length.
 | Value | Description |
 |-------|-------------|
 | auto | VLLM calculates optimal length |
 | 2048 | 2048 tokens |
 | 4096 | 4096 tokens |
 | 8192 | 8192 tokens |
 | 32768 | 32768 tokens |
 Smaller values reduce memory usage but limit context length.
 ### max_num_batched_tokens
 This parameter sets the maximum number of tokens per batch.
 | Value | Description |
 |-------|-------------|
 | 2048 | Default for online serving |
 | 4096 | Moderate batch size |
 | 8192 | Large batch size |
 | 16384 | Very large batch size |
 ### max_num_seqs
 This parameter sets the maximum number of sequences per batch.
 | Value | Description |
 |-------|-------------|
 | 128 | Default for online serving |
 | 256 | Default for offline inference |
 | 512 | Large batch |
 ### kv_cache_dtype
 This parameter sets the KV cache data type.
 | Value | Description |
 |-------|-------------|
 | auto | Uses model default dtype |
 | fp8 | Uses FP8 for KV cache (reduces memory by 50%) |
 | fp8_e5m2 | Uses FP8 E5M2 format for KV cache |
 ## Memory Configuration Examples
 ### Small Model on Single GPU
 ```
 vllm serve Qwen/Qwen2.5-1.5B-Instruct \
    --gpu-memory-utilization 0.9 \
    --swap-space 4 \
    --max-num-batched-tokens 4096 \
    --max-num-seqs 128
 ```
 ### Large Model with CPU Offload
 ```
 vllm serve meta-llama/Llama-3.1-70B-Instruct \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.8 \
    --swap-space 16 \
    --max-model-len 8192
 ```
 ### Memory-Constrained GPU
 ```
 vllm serve meta-llama/Llama-3.1-70B-Instruct \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.7 \
    --swap-space 32 \
    --max-model-len 4096 \
    --max-num-batched-tokens 2048
 ```
 ## Preemption Modes
 VLLM uses two preemption modes when GPU memory is insufficient:
 ### Recompute Mode (Default in V1)
 VLLM recomputes the prefill for preempted requests. This uses GPU compute instead of memory. This is the default mode in VLLM V1.
 ### Swap Mode
 VLLM swaps KV cache pages to CPU and reloads them later. This uses CPU memory instead of compute.
 ## Monitoring
 Watch for preemption warnings:
 ```
 WARNING scheduler.py: Sequence group 0 is preempted by PreemptionMode.RECOMPUTE mode because there is not enough KV cache space.
 ```
 If you see frequent preemptions:
 1. Increase `gpu_memory_utilization`
 2. Increase `swap_space`
 3. Decrease `max_num_seqs`
 4. Increase `tensor_parallel_size`
 5. Increase `max_model_len` to match actual usage
 ## Memory Conservation Tips
 - Use quantization (FP8, INT8) to reduce model size
 - Reduce `max_model_len` to match actual needs
 - Reduce `max_num_seqs` to limit concurrent requests
 - Disable CUDA graphs with `--enforce-eager` to save memory
 - Set `compilation_config.cudagraph_capture_sizes` to smaller values
 - Use `--kv-cache-memory` to skip memory profiling
 - Limit multi-modal inputs with `limit_mm_per_prompt`
 ## Environment Variables
 | Variable | Description | Default |
 |----------|-------------|---------|
 | VLLM_CPU_KVCACHE_SPACE | KV cache space in GiB for CPU backend | 4 |
 | VLLM_CPU_OMP_THREADS_BIND | CPU thread binding for OpenMP | auto |
 | VLLM_CPU_NUM_OF_RESERVED_CPU | Reserved CPU cores per rank | 0-1 |
 | VLLM_CACHE_ROOT | torch.compile cache directory | ~/.cache/vllm |
 ## See Also
 - [[VLLM Overview]]
 - [[VLLM Index]]
 - [[Hybrid CPU GPU Processing Dankserver]]
 - [[Shared Memory Architecture Dankclaw]]
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Frank	1f85e57863	Update AGENTS.md with target-specific branching and service status	2026-08-09 10:37:57 -07:00
Frank	6bddf0a72a	Add AGENTS.md, deploy script, update README with service status and file locations	2026-08-09 09:47:24 -07:00
Frank	469bfaf979	Update README with validation instructions and performance data	2026-08-08 21:46:00 -07:00
Frank	726b9a35ec	Add validator script with target selection and test suite	2026-08-08 21:42:37 -07:00
Frank	5ed9bf0410	Update documentation with AWQ+DFlash lessons learned and improved validation script	2026-08-08 21:29:23 -07:00
Frank	c530e16880	Temporarily disable speculative decoding to test base model	2026-08-08 17:28:00 -07:00
Frank	9d00f36d2b	Lower context to 32k, fix duplicate lines, remove prefix caching	2026-08-08 17:03:16 -07:00
Frank	9227e1a8b9	Add AWQ DFlash Ampere script and improved validation - vllm-awq-qwen36-dflash-ampere.sh: AWQ-4bit Qwen3.6-35B-A3B with DFlash speculative decoding, optimized for Ampere hardware - Model: cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit (AWQ Marlin) - Draft: z-lab/Qwen3.6-35B-A3B-DFlash - --max-model-len 32768 - --max-num-batched-tokens 1536 - --cpu-offload-gb 40 (experts) - --reasoning-parser qwen3 - --tool-call-parser qwen3_xml - Port: 34401 - Note: prefix caching disabled (Mamba cache align conflict) - scripts/validate-vllm.sh: Per-stream latency tests - Separate tests for 1, 2, 3, 4 streams - Different prompts per stream to avoid caching effects - Reports elapsed ms per stream count	2026-08-08 16:41:44 -07:00
Frank	659fba6a5d	Lower --max-num-batched-tokens to 1536 to free VRAM for weights	2026-08-08 12:56:28 -07:00
Frank	9c1f777423	Fix serve-vllm-qwen36-dflash.sh: remove 'vllm serve' prefix (image entrypoint handles it)	2026-08-08 12:15:31 -07:00
Frank	4edcada2a8	Increase max-model-len to 32768, cpu-offload to 40GB, add reasoning-parser and tool-call-parser	2026-08-08 12:03:23 -07:00