Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Frank
|
1f85e57863 | ||
|
Frank
|
6bddf0a72a | ||
|
Frank
|
469bfaf979 | ||
|
Frank
|
726b9a35ec | ||
|
Frank
|
5ed9bf0410 | ||
|
Frank
|
c530e16880 | ||
|
Frank
|
9d00f36d2b | ||
|
Frank
|
9227e1a8b9 | ||
|
Frank
|
659fba6a5d | ||
|
Frank
|
9c1f777423 | ||
|
Frank
|
4edcada2a8 |
@@ -1,71 +0,0 @@
|
||||
# Tunables for frank-default-small (Tiel-Coder-35B-A3B UD-Q6_K_XL,
|
||||
# qwen35moe hybrid MoE, Dankserver, RTX 3090).
|
||||
#
|
||||
# Runtime: upstream llama.cpp (latest master, CUDA build with
|
||||
# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86) on RTX 3090
|
||||
# 24 GB; Ryzen 7 7700X; 62 GB RAM. Replaced the ik_llama.cpp Ornith
|
||||
# CRACK Q8_0 deployment on 2026-09-01.
|
||||
#
|
||||
# Model: peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF, UD-Q6_K_XL quant
|
||||
# (30 GB). Architecture: qwen35moe — 40 layers, full_attention_interval=4
|
||||
# (10 attention + 30 SSM/Mamba), 256 experts / 8 used, 2 KV heads × 256
|
||||
# dim, embedding 2048, rope dimension_count 64, rope freq_base 10 000 000.
|
||||
# Text-only (no mmproj). Built-in chat template in the GGUF.
|
||||
#
|
||||
# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
|
||||
# Non-expert weights (~1.6 GB) fit entirely on the 24 GB card.
|
||||
#
|
||||
# KV cache: Q8_0 for both K and V. With CTX_SIZE=200700 (66 900 per slot
|
||||
# × 3 parallel) and 10 attention layers, the KV cache is ~2 GB — the
|
||||
# requested 2 GB VRAM buffer.
|
||||
#
|
||||
# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
|
||||
# ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
|
||||
# incident). This box is NVIDIA/CUDA, but the rule keeps the env portable.
|
||||
|
||||
# Model file (UD-Q6_K_XL, 30 GB, text-only MoE hybrid).
|
||||
MODEL_DIR=/home/dank/models/tiel-coder-35b
|
||||
MODEL_FILE=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
|
||||
|
||||
# Upstream llama.cpp llama-server (CUDA build, latest master).
|
||||
# Built with -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86
|
||||
# using the staged CUDA 13.2 toolkit at /home/dank/cuda13-stage.
|
||||
LLAMA_SERVER=/home/dank/llama.cpp/build/bin/llama-server
|
||||
|
||||
ALIAS=frank-default-small
|
||||
BIND_ADDR=0.0.0.0
|
||||
LISTEN_PORT=7770
|
||||
|
||||
# Total context budget across all parallel slots. 200700 / 3 = 66 900
|
||||
# per slot. The model's context_length is 262 144, so 66 900 is well
|
||||
# within range.
|
||||
CTX_SIZE=200700
|
||||
PARALLEL=3
|
||||
|
||||
# Expert offload: all MoE expert weights pinned to CPU RAM.
|
||||
CPU_MOE=1
|
||||
|
||||
# All non-expert layers on GPU (auto-fit will determine the exact count;
|
||||
# with --cpu-moe the non-expert footprint is ~1.6 GB, leaving ample room).
|
||||
N_GPU_LAYERS=999
|
||||
|
||||
# KV cache: Q8_0 for both K and V. ~2 GB total for 200700 ctx × 3 slots
|
||||
# × 10 attention layers × (2 KV heads × 256 dim × 1 byte) × 2 (K+V).
|
||||
CACHE_TYPE_K=q8_0
|
||||
CACHE_TYPE_V=q8_0
|
||||
|
||||
VERBOSITY=2
|
||||
|
||||
# Prefill batch sizing.
|
||||
BATCH_SIZE=4096
|
||||
UBATCH_SIZE=1024
|
||||
|
||||
# Sampling.
|
||||
TEMPERATURE=1.0
|
||||
TOP_K=20
|
||||
TOP_P=0.95
|
||||
MIN_P=0.0
|
||||
|
||||
# Compute threads lowered so other processes keep a core.
|
||||
THREADS=7
|
||||
THREADS_HTTP=8
|
||||
@@ -1,16 +0,0 @@
|
||||
[Unit]
|
||||
Description=llama.cpp Tiel-Coder-35B-A3B Q6_K_XL qwen35moe hybrid MoE (dankserver, frank-default-small)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-small.env"
|
||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,126 +0,0 @@
|
||||
# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
|
||||
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
|
||||
# Strix Halo gfx1151, unified memory).
|
||||
#
|
||||
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
||||
# qwen4exp-on-rocmfpx-d3ca537 patch applied — the model card's pinned
|
||||
# runtime: ROCmFP4 tensor types AND the qwen4exp architecture in one
|
||||
# tree; stock llama.cpp and charlie's fork alone cannot load this GGUF).
|
||||
# Built per the card recipe:
|
||||
# cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
|
||||
# at /home/dank/GitPackages/kingjones-ROCmFPX/build.
|
||||
# Replaced the agentionai ROCmFP4-FAST imatrix deployment (charlie's
|
||||
# build-qwen38 runtime) on 2026-08-30.
|
||||
#
|
||||
# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
|
||||
# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
|
||||
# at layer 1), Q4_0_ROCMFP4_STRIX (file_type 105), 113.47 GiB single
|
||||
# file. STRIX is the top quality tier: PLE table at Q8_0 (50.66 GiB),
|
||||
# token embeddings + lm head at Q6_K. The PLE table NEVER enters GPU
|
||||
# memory — it streams from the file through the OS page cache. Two hard
|
||||
# card constraints:
|
||||
# 1. mmap MUST stay enabled (default) — --no-mmap forces the 50.66 GiB
|
||||
# table into anonymous memory and the cgroup killer takes the
|
||||
# process with nothing in the server log (only dmesg).
|
||||
# 2. Do NOT pin the PLE tensor to CPU with -ot — measured gen fell
|
||||
# ~23 -> 13.4 tok/s; the kernel streams it better.
|
||||
# NOTE: the card's own build_attn_qsa still calls ggml_fill on the
|
||||
# F16-cast kq mask (the path that aborted our charlie build with
|
||||
# --flash-attn on before scripts/patches/qwen4exp-qsa-f32-mask.patch).
|
||||
# If a clean kingjones-tree build aborts graph build with
|
||||
# GGML_ASSERT(a-type == GGML_TYPE_F32), port that fix (F32 base mask +
|
||||
# cast at the attention call) into this tree and rebuild.
|
||||
#
|
||||
# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
|
||||
# incident). llama.cpp reads --host/--port/--temp as CLI flags, never as
|
||||
# env vars.
|
||||
|
||||
# kingjones30/ROCmFPX runtime binary (card recipe build).
|
||||
LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
|
||||
|
||||
# Model package from
|
||||
# https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
|
||||
# Layout: single unsharded GGUF, text-only (the repo ships no mmproj;
|
||||
# the vision tower below is borrowed from the agentionai FAST package —
|
||||
# same base model, f16 — kept only if it loads clean, drop
|
||||
# MMPROJ_PATH if it does not).
|
||||
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
|
||||
MODEL_FILE=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
|
||||
MMPROJ_PATH=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf
|
||||
|
||||
# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
|
||||
# Sharp template drives thinking on by default (medium effort); the server
|
||||
# emits reasoning in DeepSeek format so clients receive reasoning_content
|
||||
# separately instead of literal thinking tags in content.
|
||||
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
||||
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
||||
|
||||
# Alias + network.
|
||||
ALIAS=frank-dragoon
|
||||
BIND_ADDR=0.0.0.0
|
||||
LISTEN_PORT=7770
|
||||
|
||||
# Serving: 401408-token context, TWO slots (parallel 2), batch 4096 /
|
||||
# ubatch 512, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
|
||||
# prompt cache. --fit off: auto-fit segfaults on this model's
|
||||
# hyper-connection/indexer config; -ngl 999 puts every layer on the
|
||||
# SVM-backed GPU pool. ubatch 512 matches the card's re-measurement
|
||||
# (-ub 1024/2048/4096 showed no effect; -t 32 vs 16 is noise).
|
||||
#
|
||||
# CTX_SIZE=401408 / PARALLEL=2: carried over from the FAST deployment
|
||||
# where it loaded clean (~74 s, health 200). The card measures the full
|
||||
# 262144 window at ~71.7 GiB GTT at load on the same 128 GB board; QSA
|
||||
# caps KV growth so the extra window is cheap. KV footprint scales
|
||||
# with the 12 attention layers only. Verify load on first STRIX boot.
|
||||
CTX_SIZE=401408
|
||||
PARALLEL=2
|
||||
|
||||
BATCH_SIZE=4096
|
||||
UBATCH_SIZE=512
|
||||
NUM_THREADS=16
|
||||
THREAD_BATCH=16
|
||||
CACHE_TYPE_K=f16
|
||||
CACHE_TYPE_V=f16
|
||||
CTX_CHECKPOINTS=16
|
||||
CACHE_RAM=8192
|
||||
SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
|
||||
SPLIT_MODE=none
|
||||
FIT=off
|
||||
NGL=999
|
||||
FLASH_ATTN=on
|
||||
|
||||
# Sampling (card instruct table, non-thinking).
|
||||
TEMPERATURE=0.7
|
||||
TOP_K=20
|
||||
TOP_P=0.8
|
||||
MIN_P=0.0
|
||||
PRESENCE_PENALTY=1.5
|
||||
|
||||
# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
|
||||
# keep the reasoning trail in the conversation). DeepSeek format projects the
|
||||
# reasoning into the API's reasoning_content field instead of literal thinking
|
||||
# tags.
|
||||
REASONING=on
|
||||
REASONING_FORMAT=deepseek
|
||||
|
||||
# Strix Halo unified memory + native gfx1151.
|
||||
HSA_OVERRIDE_GFX_VERSION=11.5.1
|
||||
|
||||
VERBOSITY=4
|
||||
|
||||
|
||||
# Speculative decoding: disabled. The card's Usage block ships no spec
|
||||
# flags. The Flash-Next MTP heads do not load on this loader (requires
|
||||
# per-layer blk.N.hc_attn_norm; every head carries only blk.48 —
|
||||
# verified 3x on build-qwen38). DRAFT_MODEL_* point at the retired
|
||||
# agentionai FAST head and are unused while SPEC_TYPE=none.
|
||||
DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
|
||||
DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
|
||||
SPEC_TYPE=none
|
||||
SPEC_DRAFT_N_MAX=3
|
||||
SPEC_DRAFT_N_MIN=0
|
||||
SPEC_DRAFT_P_MIN=0.75
|
||||
SPEC_DRAFT_NGL=999
|
||||
SPEC_DRAFT_DEVICE=ROCm0
|
||||
SPEC_DRAFT_TYPE_K=f16
|
||||
SPEC_DRAFT_TYPE_V=f16
|
||||
@@ -1,16 +0,0 @@
|
||||
[Unit]
|
||||
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,23 +0,0 @@
|
||||
# Tunables for frank-npu (FastFlowLM NPU endpoint, Dankclaw)
|
||||
# Runtime: FLM 1.0.1 (/usr/bin/flm) on the XDNA2 NPU (50 TOPS). Boot tag is
|
||||
# nuextract3:4b (NuExtract3-4B-NPU2, Qwen3.5-4B skeleton, structured-extraction
|
||||
# model); serve routes every request by the exact FLM registry tag in the
|
||||
# request's model field, so any registered tag (gemma4-it:e4b, qwythos:9b, ...)
|
||||
# is reachable on this port regardless of the boot tag. The NPU is a
|
||||
# single-lane device: generations serialize FIFO, and the first request after a
|
||||
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
||||
# pool shared with frank-dragoon (iGPU).
|
||||
FLM_MODEL=nuextract3:4b
|
||||
FLM_HOST=0.0.0.0
|
||||
FLM_PORT=7771
|
||||
# turbo = max NPU power mode (xrt-smi configure --pmode turbo at serve start).
|
||||
# No other tenant uses the NPU on this host, so the top mode costs nothing to
|
||||
# neighbours; drop to `performance` if iGPU power-budget contention ever shows.
|
||||
FLM_PMODE=turbo
|
||||
# FLM_ASR=1 additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes
|
||||
# /v1/audio/transcriptions. NuExtract3 answers text-only; audio stays off.
|
||||
FLM_ASR=0
|
||||
# -1 = no --ctx-len flag: every pack loads with its own trained context
|
||||
# (nuextract3:4b -> 32768, gemma4-it:e4b -> 131072). A global override would
|
||||
# over-allocate KV for the small packs.
|
||||
FLM_CTX_LEN=-1
|
||||
@@ -1,17 +0,0 @@
|
||||
[Unit]
|
||||
Description=FastFlowLM NPU sight+sound inference (frank-npu) on Dankclaw
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
Environment="FRANK_NPU_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-npu.env"
|
||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-npu.sh
|
||||
LimitMEMLOCK=infinity
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,180 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
|
||||
# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
|
||||
# Dankclaw (Strix Halo gfx1151, unified memory).
|
||||
#
|
||||
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
||||
# model card's qwen4exp-on-rocmfpx patch) built at
|
||||
# /home/dank/GitPackages/kingjones-ROCmFPX/build (HIP, gfx1151,
|
||||
# ROCm 7.2.4, card recipe). The ROCmFP4 quant types plus the qwen4exp
|
||||
# architecture must be in one tree; stock llama.cpp cannot load this
|
||||
# GGUF.
|
||||
#
|
||||
# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
|
||||
# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
|
||||
# at layer 1) quantized Q4_0_ROCMFP4_STRIX, 113.47 GiB single file. The
|
||||
# 50.66 GiB PLE n-gram table stays OUT of GPU memory and streams via
|
||||
# mmap from the OS page cache — never add --no-mmap or -ot for it.
|
||||
#
|
||||
# Profile: card sampling table (temp 0.7, top_p 0.80, top_k 20, min_p 0.0,
|
||||
# presence 1.5 — non-thinking instruct) mixed with the validated serving
|
||||
# profile: ctx 401408 x 2 slots, b 4096 / ub 512, t 16 / tb 16, f16 KV,
|
||||
# fa on (if the graph build aborts on the QSA F16 mask, see the note in
|
||||
# the env file: port scripts/patches/qwen4exp-qsa-f32-mask.patch into
|
||||
# this tree and rebuild), cont-batching, prompt cache + 8 GiB cache-ram
|
||||
# + idle-slot cache, 16 ctx checkpoints, --fit off (auto-fit segfaults
|
||||
# on qwen4exp hyper-connection config). Speculative decoding off: the
|
||||
# Flash-Next MTP heads do not load on this loader; the card ships no
|
||||
# spec flags.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||
if [[ -f "$ENV_FILE" ]]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
source "$ENV_FILE"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${LLAMA_BIN:=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
|
||||
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
|
||||
: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
|
||||
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
|
||||
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
||||
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
||||
: "${ALIAS:=frank-dragoon}"
|
||||
: "${BIND_ADDR:=0.0.0.0}"
|
||||
: "${LISTEN_PORT:=7770}"
|
||||
: "${CTX_SIZE:=401408}"
|
||||
: "${PARALLEL:=2}"
|
||||
: "${BATCH_SIZE:=4096}"
|
||||
: "${UBATCH_SIZE:=512}"
|
||||
: "${NUM_THREADS:=16}"
|
||||
: "${THREAD_BATCH:=16}"
|
||||
: "${CACHE_TYPE_K:=f16}"
|
||||
: "${CACHE_TYPE_V:=f16}"
|
||||
: "${CTX_CHECKPOINTS:=16}"
|
||||
: "${CACHE_RAM:=8192}"
|
||||
: "${SLOT_SAVE_PATH:=$SCRIPT_DIR/slot-state}"
|
||||
: "${SPLIT_MODE:=none}"
|
||||
: "${FIT:=off}"
|
||||
: "${NGL:=999}"
|
||||
: "${FLASH_ATTN:=on}"
|
||||
: "${TEMPERATURE:=0.7}"
|
||||
: "${TOP_K:=20}"
|
||||
: "${TOP_P:=0.8}"
|
||||
: "${MIN_P:=0.0}"
|
||||
: "${PRESENCE_PENALTY:=1.5}"
|
||||
: "${REASONING:=on}"
|
||||
: "${REASONING_FORMAT:=deepseek}"
|
||||
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
|
||||
: "${SPEC_TYPE:=none}"
|
||||
: "${DRAFT_MODEL_DIR:=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST}"
|
||||
: "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
|
||||
: "${SPEC_DRAFT_N_MAX:=3}"
|
||||
: "${SPEC_DRAFT_N_MIN:=0}"
|
||||
: "${SPEC_DRAFT_P_MIN:=0.75}"
|
||||
: "${SPEC_DRAFT_NGL:=999}"
|
||||
: "${SPEC_DRAFT_DEVICE:=ROCm0}"
|
||||
: "${SPEC_DRAFT_TYPE_K:=f16}"
|
||||
: "${SPEC_DRAFT_TYPE_V:=f16}"
|
||||
|
||||
MODEL="$MODEL_DIR/$MODEL_FILE"
|
||||
for f in "$LLAMA_BIN" "$MODEL" "$CHAT_TEMPLATE_FILE"; do
|
||||
if [[ ! -r "$f" ]]; then
|
||||
echo "missing or unreadable required artifact: $f" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
if [[ ! -x "$LLAMA_BIN" ]]; then
|
||||
echo "missing executable runtime: $LLAMA_BIN" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- speculative decoding (MTP draft head) ---------------------------------
|
||||
# The MTP draft head is a separate GGUF sidecar loaded via --spec-draft-model
|
||||
# (alias --model-draft). ngram-only specs do not need a draft file.
|
||||
# Set SPEC_TYPE=none in the env to disable speculative decoding entirely.
|
||||
SPEC_ARGS=()
|
||||
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
||||
SPEC_ARGS=(
|
||||
--spec-type "$SPEC_TYPE"
|
||||
--spec-draft-n-max "$SPEC_DRAFT_N_MAX"
|
||||
--spec-draft-n-min "$SPEC_DRAFT_N_MIN"
|
||||
--spec-draft-p-min "$SPEC_DRAFT_P_MIN"
|
||||
)
|
||||
if [[ "$SPEC_TYPE" == *draft-mtp* ]]; then
|
||||
DRAFT_MODEL="$DRAFT_MODEL_DIR/$DRAFT_MODEL_FILE"
|
||||
if [[ ! -r "$DRAFT_MODEL" ]]; then
|
||||
echo "missing or unreadable draft model: $DRAFT_MODEL" >&2
|
||||
exit 1
|
||||
fi
|
||||
SPEC_ARGS+=(
|
||||
--spec-draft-model "$DRAFT_MODEL"
|
||||
--spec-draft-ngl "$SPEC_DRAFT_NGL"
|
||||
--spec-draft-device "$SPEC_DRAFT_DEVICE"
|
||||
--spec-draft-type-k "$SPEC_DRAFT_TYPE_K"
|
||||
--spec-draft-type-v "$SPEC_DRAFT_TYPE_V"
|
||||
)
|
||||
fi
|
||||
if [[ -n "${SPEC_DRAFT_P_SPLIT:-}" ]]; then
|
||||
SPEC_ARGS+=(--spec-draft-p-split "$SPEC_DRAFT_P_SPLIT")
|
||||
fi
|
||||
fi
|
||||
|
||||
# The STRIX repo ships no mmproj; this is the agentionai FAST package's
|
||||
# f16 vision tower for the same base model. Passed only when readable.
|
||||
MMPROJ_ARGS=()
|
||||
if [[ -n "${MMPROJ_PATH:-}" && -r "${MMPROJ_PATH:-}" ]]; then
|
||||
MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
|
||||
fi
|
||||
|
||||
SERVER_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
mkdir -p "$SLOT_SAVE_PATH"
|
||||
ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
|
||||
LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
||||
|
||||
exec /usr/bin/env \
|
||||
ROCM_PATH="$ROCM_PATH" \
|
||||
HIP_VISIBLE_DEVICES=0 \
|
||||
LD_LIBRARY_PATH="$LIBRARY_PATH" \
|
||||
HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
|
||||
"$LLAMA_BIN" \
|
||||
--model "$MODEL" \
|
||||
--alias "$ALIAS" \
|
||||
--chat-template-file "$CHAT_TEMPLATE_FILE" \
|
||||
--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS" \
|
||||
--host "$BIND_ADDR" \
|
||||
--port "$LISTEN_PORT" \
|
||||
--jinja \
|
||||
--device ROCm0 \
|
||||
--gpu-layers "$NGL" \
|
||||
--split-mode "$SPLIT_MODE" \
|
||||
--parallel "$PARALLEL" \
|
||||
--ctx-size "$CTX_SIZE" \
|
||||
--batch-size "$BATCH_SIZE" \
|
||||
--ubatch-size "$UBATCH_SIZE" \
|
||||
--fit "$FIT" \
|
||||
--flash-attn "$FLASH_ATTN" \
|
||||
--cache-type-k "$CACHE_TYPE_K" \
|
||||
--cache-type-v "$CACHE_TYPE_V" \
|
||||
--threads "$NUM_THREADS" \
|
||||
--threads-batch "$THREAD_BATCH" \
|
||||
--ctx-checkpoints "$CTX_CHECKPOINTS" \
|
||||
--cache-ram "$CACHE_RAM" \
|
||||
--slot-save-path "$SLOT_SAVE_PATH" \
|
||||
--cache-prompt \
|
||||
--cache-idle-slots \
|
||||
--metrics \
|
||||
--temp "$TEMPERATURE" \
|
||||
--top-p "$TOP_P" \
|
||||
--top-k "$TOP_K" \
|
||||
--min-p "$MIN_P" \
|
||||
--presence-penalty "$PRESENCE_PENALTY" \
|
||||
--reasoning "$REASONING" \
|
||||
--reasoning-format "$REASONING_FORMAT" \
|
||||
--reasoning-budget -1 \
|
||||
"${MMPROJ_ARGS[@]}" \
|
||||
"${SPEC_ARGS[@]}"
|
||||
@@ -1,62 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve the FastFlowLM (FLM) sight+sound endpoint — frank-npu.
|
||||
#
|
||||
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
|
||||
# single-lane device: generations serialize in FIFO order, so this is a
|
||||
# background/triage endpoint, not an interactive multi-user chat. The NPU
|
||||
# draws from the unified RAM pool shared with frank-dragoon (the iGPU
|
||||
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
|
||||
# for the memory budget and the model-selection tradeoff.
|
||||
#
|
||||
# NOTE: FLM requires unlimited memlock. The unit sets LimitMEMLOCK=infinity.
|
||||
# NOTE: an unknown model tag does NOT 404 — /v1/chat/completions answers HTTP 200 with a
|
||||
# flat {"error":"Failed to load <tag> model!"} body and no choices; /v1/completions returns a
|
||||
# nested OpenAI-shaped server_error (code 500) also under HTTP 200. Client model IDs must be the exact FLM registry
|
||||
# tag (e.g. gemma4-it:e4b), never a service alias. Custom tags only resolve
|
||||
# when FLM_CONFIG_PATH points at the user registry that carries them and their
|
||||
# kernels are symlinked into /home/dank/.config/flm/xclbins/<PackName>.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ENV_FILE="${FRANK_NPU_ENV_FILE:-$SCRIPT_DIR/frank-npu.env}"
|
||||
if [[ -f "$ENV_FILE" ]]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
source "$ENV_FILE"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${FLM_BIN:=/usr/bin/flm}"
|
||||
: "${FLM_MODEL:=gemma4-it:e4b}"
|
||||
: "${FLM_HOST:=0.0.0.0}"
|
||||
: "${FLM_PORT:=7771}"
|
||||
: "${FLM_PMODE:=turbo}"
|
||||
: "${FLM_ASR:=0}"
|
||||
: "${FLM_CTX_LEN:=131072}"
|
||||
|
||||
# The user registry is a superset of the system registry (it carries the
|
||||
# custom tags too), and its model_path resolves relative to the registry file,
|
||||
# so exports keep a tag swap in the env file from needing script edits.
|
||||
export FLM_CONFIG_PATH="${FLM_CONFIG_PATH:-/home/dank/.config/flm/model_list.json}"
|
||||
export FLM_XCLBIN_PATH="${FLM_XCLBIN_PATH:-/home/dank/.config/flm/xclbins}"
|
||||
|
||||
if [[ ! -x "$FLM_BIN" ]]; then
|
||||
echo "missing flm binary: $FLM_BIN" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- optional flags ----------------------------------------------------------
|
||||
OPT_ARGS=()
|
||||
if [[ "${FLM_ASR}" != "0" ]]; then
|
||||
OPT_ARGS+=(--asr "${FLM_ASR}")
|
||||
fi
|
||||
if [[ "${FLM_CTX_LEN}" != "-1" ]]; then
|
||||
OPT_ARGS+=(--ctx-len "${FLM_CTX_LEN}")
|
||||
fi
|
||||
|
||||
exec "$FLM_BIN" serve "$FLM_MODEL" \
|
||||
--host "$FLM_HOST" \
|
||||
-p "$FLM_PORT" \
|
||||
--pmode "$FLM_PMODE" \
|
||||
"${OPT_ARGS[@]}"
|
||||
@@ -1,87 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve peculiar-ragdoll/Tiel-Coder-35B-A3B (UD-Q6_K_XL) as
|
||||
# frank-default-small via stock llama.cpp on Dankserver (RTX 3090).
|
||||
#
|
||||
# Runtime: upstream llama.cpp (latest master, CUDA build with
|
||||
# -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86).
|
||||
# The model is a 35B-total / 8-active MoE hybrid (qwen35moe arch) with
|
||||
# SSM/Mamba layers (full_attention_interval=4: 10 attention + 30 SSM
|
||||
# layers out of 40). 256 experts, 8 used per token.
|
||||
#
|
||||
# Expert offload: --cpu-moe keeps all 256 expert weights in CPU RAM.
|
||||
# Non-expert weights (attention, SSM, shared expert, embeddings, output)
|
||||
# are ~1.6 GB and fit entirely on the 24 GB RTX 3090.
|
||||
#
|
||||
# KV cache: Q8_0 for both K and V. With CTX_SIZE=200700 (66 900 per
|
||||
# slot × 3 parallel) and 10 attention layers (2 KV heads × 256 dim),
|
||||
# the KV cache is ~2 GB — the requested 2 GB VRAM buffer.
|
||||
#
|
||||
# Model is loaded with mmap (default). The 30 GB Q6_K_XL file is
|
||||
# memory-mapped; only accessed pages enter RAM.
|
||||
#
|
||||
# Tunables come from frank-default-small.env unless FRANK_SMALL_ENV_FILE
|
||||
# overrides that path.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ENV_FILE="${FRANK_SMALL_ENV_FILE:-$SCRIPT_DIR/frank-default-small.env}"
|
||||
if [[ -f "$ENV_FILE" ]]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
source "$ENV_FILE"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -----------------
|
||||
: "${MODEL_DIR:=/home/dank/models/tiel-coder-35b}"
|
||||
: "${MODEL_FILE:=Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf}"
|
||||
: "${LLAMA_SERVER:=/home/dank/llama.cpp/build/bin/llama-server}"
|
||||
: "${ALIAS:=frank-default-small}"
|
||||
: "${BIND_ADDR:=0.0.0.0}"
|
||||
: "${LISTEN_PORT:=7770}"
|
||||
: "${CTX_SIZE:=200700}"
|
||||
: "${PARALLEL:=3}"
|
||||
: "${CACHE_TYPE_K:=q8_0}"
|
||||
: "${CACHE_TYPE_V:=q8_0}"
|
||||
: "${N_GPU_LAYERS:=999}"
|
||||
: "${BATCH_SIZE:=4096}"
|
||||
: "${UBATCH_SIZE:=1024}"
|
||||
: "${VERBOSITY:=2}"
|
||||
: "${CPU_MOE:=1}"
|
||||
: "${TEMPERATURE:=1.0}"
|
||||
: "${TOP_K:=20}"
|
||||
: "${TOP_P:=0.95}"
|
||||
: "${MIN_P:=0.0}"
|
||||
: "${THREADS:=7}"
|
||||
: "${THREADS_HTTP:=8}"
|
||||
|
||||
MODEL="$MODEL_DIR/$MODEL_FILE"
|
||||
|
||||
if [[ ! -f "$MODEL" ]]; then
|
||||
echo "missing required file: $MODEL" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ ! -x "$LLAMA_SERVER" ]]; then
|
||||
echo "missing llama-server binary: $LLAMA_SERVER" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exec "$LLAMA_SERVER" \
|
||||
--model "$MODEL" \
|
||||
$([ "$CPU_MOE" = "1" ] && printf -- "--cpu-moe") \
|
||||
--cache-type-k "$CACHE_TYPE_K" \
|
||||
--cache-type-v "$CACHE_TYPE_V" \
|
||||
--ctx-size "$CTX_SIZE" \
|
||||
--parallel "$PARALLEL" \
|
||||
--gpu-layers "$N_GPU_LAYERS" \
|
||||
--batch-size "$BATCH_SIZE" \
|
||||
--ubatch-size "$UBATCH_SIZE" \
|
||||
--temp "$TEMPERATURE" \
|
||||
--top-k "$TOP_K" \
|
||||
--top-p "$TOP_P" \
|
||||
--min-p "$MIN_P" \
|
||||
--threads "$THREADS" \
|
||||
--threads-http "$THREADS_HTTP" \
|
||||
--alias "$ALIAS" \
|
||||
--host "$BIND_ADDR" \
|
||||
--port "$LISTEN_PORT"
|
||||
@@ -1,6 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Repository pre-commit hook: run only tests selected by staged paths.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
exec "$REPO_ROOT/scripts/test-changed.sh"
|
||||
-34
@@ -1,40 +1,6 @@
|
||||
# Local environment files — never commit real values
|
||||
.env
|
||||
*.env
|
||||
!.env.example
|
||||
# LLM fleet tunables (paths and numbers, never credentials)
|
||||
!scripts/*.env
|
||||
|
||||
# LLM fleet runtime state (slot save/restore files written by the server)
|
||||
scripts/slot-state/
|
||||
|
||||
# Playwright e2e: local creds, browser storage states, node artifacts
|
||||
tests/e2e/.env.local
|
||||
tests/e2e/.auth/
|
||||
tests/e2e/node_modules/
|
||||
tests/e2e/test-results/
|
||||
tests/e2e/playwright-report/
|
||||
|
||||
# pi subagent artifacts
|
||||
.pi-subagents/
|
||||
|
||||
# doc-audit scratch (transient, synthesized into serverAppConfigs-docs/ then trashed)
|
||||
.docaudit/
|
||||
|
||||
# Agent trash
|
||||
.AGENTTRASH/
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
|
||||
# Obsidian editor state
|
||||
.obsidian/
|
||||
|
||||
# Editor/OS noise
|
||||
*.swp
|
||||
.DS_Store
|
||||
third_party/
|
||||
|
||||
@@ -1,843 +1,103 @@
|
||||
# AGENTS.md — serverAppConfigs Agent Rules
|
||||
# vLLM Scripts — Agent Instructions
|
||||
|
||||
## Overview
|
||||
|
||||
This repository is the single source of truth for the two homelab servers and
|
||||
the FrankBook-Pro Mac mini. It tracks the Podman container stacks and the LLM
|
||||
inference fleet. One repo, one checkout per host, one deploy script. All
|
||||
services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
|
||||
merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
|
||||
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
|
||||
Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
|
||||
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
||||
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
||||
Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
|
||||
container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
|
||||
RTX 3090). Caddy on
|
||||
Dankclaw reverse-proxies both hosts.
|
||||
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
||||
(Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
|
||||
its podman machine; all egress rides the Gluetun WireGuard netns. Deployed
|
||||
via the `frankbook-pro` handler.
|
||||
|
||||
## Fleet topology (ACTIVE)
|
||||
|
||||
| Target | Service | Runtime | Payload | Port |
|
||||
|--------|---------|---------|---------|------|
|
||||
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
|
||||
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
|
||||
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
|
||||
|
||||
Payload identity, shape, eras, laws and rollback live in ONE page each:
|
||||
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
|
||||
is the authority for every number; a page that disagrees with it is the bug.
|
||||
A retired payload is a row in that page's **Eras** table — there is no page
|
||||
per attempt and no separate swap-history file.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
|
||||
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
|
||||
- `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
|
||||
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
|
||||
scripts, systemd units, env tunables, bench harnesses
|
||||
- `old/` — retired LLM scripts (reference only)
|
||||
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
||||
The managed skill library is distilled there one page per skill-cluster:
|
||||
`LLM/index.md` → *Cross-Cutting References*, plus `Operations/index.md`,
|
||||
`Agent/index.md`, and the Infrastructure/Dankclaw/FrankBook-Pro service
|
||||
pages. There is no `log.md`: the change diary was destroyed 2026-10-19
|
||||
- `tests/` — bats suite + Playwright e2e
|
||||
|
||||
## Git Workflow — trunk-based (fleet law, aligned 2026-10-18)
|
||||
|
||||
All edits flow through git. Never edit files on a remote machine directly.
|
||||
|
||||
`main` is the only long-lived branch: no `develop`, no release branches, no
|
||||
back-merges. Frank is the only dev; this repo and `dank/command-center` follow
|
||||
the same regime.
|
||||
|
||||
1. **Branch fresh** — `git fetch`, then branch off the current `origin/main`
|
||||
(`<type>/<slug>`, type ∈ feat|fix|docs|chore). If main moves under you,
|
||||
merge it into your branch — never rebase.
|
||||
2. **Source** the live config from the target machine
|
||||
3. **Commit** the fetched version here (redact secrets into `.env`)
|
||||
4. **Edit** locally
|
||||
5. **Test** — `./tests/run.sh` must pass
|
||||
6. **Push the branch and open a PR against `main`** — origin is Gitea on
|
||||
Dankclaw. Merge with `--no-ff`; **never merge a PR without an explicit,
|
||||
current OK from Frank.** **The moment the PR is open, open it in Frank's
|
||||
browser** (`xdg-open <PR-URL>` — a Firefox tab on the workstation); never
|
||||
make him hunt for it in Gitea. Releases: tag `vX.Y.Z` straight on `main`.
|
||||
7. **Pull** on the target (`deploy.sh` does this)
|
||||
8. **Deploy** — `scripts/deploy.sh` plans and dispatches only changed services
|
||||
|
||||
**Attribution footer (issues and PRs)** — every issue or PR body you open MUST end with:
|
||||
## Project Overview
|
||||
Scripts for running large language models via vLLM on two local machines:
|
||||
- **dankserver**: NVIDIA GPU + System RAM (frank-default-small)
|
||||
- **dankclaw**: AMD Strix Halo Shared Memory (frank-default-large)
|
||||
|
||||
## Directory Structure
|
||||
```
|
||||
---
|
||||
cwd: <working directory> · session: <omp session id>
|
||||
vllmScripts/
|
||||
├── AGENTS.md # This file — agent instructions
|
||||
├── README.md # Human-readable project overview
|
||||
├── deploy.sh # Automated deployment script
|
||||
├── scripts/ # Executable vLLM launch scripts
|
||||
│ ├── validate-vllm.sh # Validation tool
|
||||
│ └── *.sh # Inference server scripts
|
||||
├── tests/ # Test suite
|
||||
│ └── test_validator.sh
|
||||
├── docs/ # Knowledge base
|
||||
│ ├── README.md
|
||||
│ └── *.md # Research notes
|
||||
└── failedAttempts/ # Archived scripts (for reference)
|
||||
```
|
||||
|
||||
Session id from `/status`; `cwd` is that session's own working directory — not the build checkout, when they differ (note the build worktree in the body instead). Machine-made artifacts stay attributable to the box and session that spawned them.
|
||||
## Running Tests
|
||||
```bash
|
||||
# Test validator on dankserver
|
||||
bash tests/test_validator.sh dankserver
|
||||
|
||||
# Test validator on dankclaw
|
||||
bash tests/test_validator.sh dankclaw
|
||||
|
||||
# Run all tests
|
||||
bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw
|
||||
```
|
||||
|
||||
## Deployment
|
||||
```bash
|
||||
# Deploy all scripts to all targets
|
||||
bash deploy.sh all
|
||||
|
||||
```
|
||||
./scripts/deploy.sh dankclaw
|
||||
./scripts/deploy.sh dankserver
|
||||
./scripts/deploy.sh dankclaw --service searxng
|
||||
./scripts/deploy.sh dankserver --service frank-archon
|
||||
./scripts/deploy.sh dankclaw --dry-run
|
||||
# Deploy specific script to dankserver
|
||||
bash deploy.sh dankserver vllm-qwen36.sh
|
||||
|
||||
# Deploy specific script to dankclaw
|
||||
bash deploy.sh dankclaw vllm-thinkingcap.sh
|
||||
```
|
||||
|
||||
The planner maps changed paths to services; the remote runner restarts only
|
||||
the selected service and records its revision outside the checkout.
|
||||
Documentation-only changes are a no-op. LLM services are selected only by
|
||||
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
||||
by its unit/env/serve trio in `scripts/` — so
|
||||
unrelated
|
||||
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
|
||||
runs on a host without passwordless sudo; the value never enters the repo.
|
||||
## Git Workflow
|
||||
|
||||
**`deploy.sh <target>` with no `--service` is a fleet-wide DRAIN, not a sync.**
|
||||
It also walks every per-service revision marker under the deploy-state dir and
|
||||
restarts each service whose marker is behind HEAD *and* whose paths changed
|
||||
since that marker — so a backlog accumulated by service-scoped deploys fires
|
||||
all at once, and one failing health gate aborts the run with every later
|
||||
service left pending. Name the service unless a drain is what you want
|
||||
(`./scripts/deploy.sh dankserver --service frank-archon`).
|
||||
### Branching Strategy
|
||||
- `main`: Stable, working scripts only
|
||||
- `dankserver/*`: Server-specific development
|
||||
- `dankserver/qwen36-awq`: Qwen3.6 AWQ experiments
|
||||
- `dankserver/*`: Feature branches for dankserver
|
||||
- `dankclaw/*`: Claw-specific development
|
||||
- `dankclaw/*`: Feature branches for dankclaw
|
||||
|
||||
### Deployment completeness (MANDATORY)
|
||||
### Worktree Pattern
|
||||
For parallel development:
|
||||
```bash
|
||||
# Create worktree for feature
|
||||
git worktree add ../vllm-feature feature/qwen36-awq
|
||||
|
||||
A deploy is **not complete** when `deploy.sh` exits green. It requires:
|
||||
|
||||
1. `./tests/run.sh` — all seams green, and
|
||||
2. the Playwright e2e suite (`tests/e2e/run.sh`) — runs **automatically** as
|
||||
the `[3/3]` gate at the end of `deploy.sh` against the LIVE fleet; a
|
||||
behavior-affecting deploy that fails it exits `DEPLOY INCOMPLETE`. Bypass
|
||||
only with `--skip-e2e` in an emergency. The gate runs the conditional
|
||||
`frankbook` Playwright project (the `*arr`/qBittorrent walk,
|
||||
`tests/e2e/specs/19-frankbook.spec.ts`) ONLY when a frankbook blast-radius
|
||||
path changed since origin/main — that spec, `frankbook-pro/**`,
|
||||
`shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the
|
||||
arr vhosts — so a sleeping FrankBook-Pro never fails an unrelated deploy.
|
||||
The default project set excludes it everywhere else (`E2E_INCLUDE_FRANKBOOK=1`
|
||||
opts in; see `serverAppConfigs-docs/Architecture/E2E Suite.md`).
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[Edit locally] --> B[Tests pass] --> C[Commit + push]
|
||||
C --> D[deploy.sh plan] --> E[Target pulls ff-only]
|
||||
E --> F[Dispatch selected service] --> G[Per-service health]
|
||||
G --> H[Playwright e2e gate] --> I{green?}
|
||||
I -->|yes| J[Deploy complete]
|
||||
I -->|no| K[DEPLOY INCOMPLETE]
|
||||
# When done, merge back
|
||||
git worktree remove ../vllm-feature
|
||||
```
|
||||
|
||||
## Boot Persistence & Self-Healing (MANDATORY)
|
||||
|
||||
Every deployed, working Podman container MUST have a self-healing,
|
||||
boot-persistent Quadlet unit. A container that runs only from `compose.yaml`
|
||||
or a manual `podman run` dies silently at reboot — rootless Podman does not
|
||||
auto-start bare containers, and `restart: unless-stopped` does not survive the
|
||||
explicit stop that shutdown performs. Never report such a service as "up".
|
||||
|
||||
Unit requirements:
|
||||
|
||||
- **Boot persistence** — `[Install] WantedBy=default.target`, installed to
|
||||
`~/.config/containers/systemd/<name>.container`, enabled in the user
|
||||
manager, with linger on for the deploy user (`loginctl show-user dank -p
|
||||
Linger` → `Linger=yes`).
|
||||
- **Self-healing** — `Restart=always` (or `on-failure`) with a short
|
||||
`RestartSec`; quadlet health check where the service has one.
|
||||
- **Repo provenance** — the `.container` file lives in this repo
|
||||
(`<host>/<svc>/quadlet/`), is installed by the deployer, and is covered by
|
||||
the quadlet test seam. The host-side unit is generated state; the repo file
|
||||
is the source of truth.
|
||||
|
||||
Before declaring a service deployed: `systemctl --user is-enabled <name>` →
|
||||
`enabled`, `is-active` → `active`, and the container answers its health check.
|
||||
A running container with no unit is a gap: write the unit, test it, deploy it.
|
||||
|
||||
This rule exists because reboots found gaps twice (2026-08-26: Caddy as a
|
||||
bare compose container never returned; 2026-09-11: five observability
|
||||
services had no units). The observability services now have units under
|
||||
`dankclaw/observability/quadlet/` and their compose definitions were removed —
|
||||
one owner per container name. The stack is **configured and delivering as of
|
||||
2026-09-17**: Grafana datasources (Prometheus and Loki) and the `llm-fleet`
|
||||
dashboard are provisioned from `shared/grafana/provisioning/`, and
|
||||
`shared/prometheus/rules/llm-fleet.yml` loads two rule groups — `llm-fleet`
|
||||
(llama.cpp/halogen, `llamacpp:*`) and `vllm-fleet` (radiance, `vllm:*`) — alert
|
||||
+ recording rules, and firing rules are
|
||||
delivered: Prometheus → Alertmanager (`dankclaw/observability/quadlet/
|
||||
alertmanager.container`, loopback :9093) → host-side Signal relay
|
||||
(`scripts/signal-relay.py`, user unit, 192.168.0.143:18093 — the host LAN
|
||||
IP, the one bind a container can reach as `host.containers.internal`;
|
||||
iptables admits loopback, the host and the podman bridge subnet only) → signal-cli account 318311.
|
||||
Prometheus targets read down for structural reasons (HTML-scraped "probes",
|
||||
Gitea metrics off); the remedy is a blackbox exporter. `frank-zealot` was down for
|
||||
a different reason until 2026-10-18: llama.cpp will not serve `/metrics` at all
|
||||
without `--metrics`. See
|
||||
`serverAppConfigs-docs/Infrastructure/Observability.md`.
|
||||
|
||||
## Testing
|
||||
|
||||
Tests are mandatory before commit (`bats` required on the dev machine):
|
||||
|
||||
```
|
||||
./tests/run.sh
|
||||
```
|
||||
|
||||
Seams: compose validity, env consistency (no leaked secrets), Caddyfile
|
||||
validity, quadlet units, docs completeness **including vault link integrity**,
|
||||
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
||||
deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
|
||||
and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
|
||||
no project, because such a spec silently never runs.
|
||||
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
||||
|
||||
## Secret Rules
|
||||
|
||||
- Never commit real secrets. The live Gitea DB password and sudo password are
|
||||
test-enforced leak checks.
|
||||
- `.env` files are git-ignored; commit only `.env.example` with placeholders.
|
||||
Exception: `scripts/*.env` LLM tunables (paths and numbers, never
|
||||
credentials).
|
||||
- Do not document passwords anywhere in the repo. SSH keys are the auth
|
||||
mechanism; `FRANK_SUDO_PW` covers headless sudo and never enters the repo.
|
||||
- **A token passed as `curl -H "Authorization: Bearer …"` is world-readable.**
|
||||
It lands in `/proc/<pid>/cmdline`, so any `ps`/`pgrep -a` on the host prints
|
||||
it — that leaked an HF token into a session transcript on 2026-09-16, while
|
||||
merely checking whether a download was still running. Pass secrets out of
|
||||
band (`curl -K <mode-600 file>` carrying the header, or `HF_TOKEN` in the
|
||||
environment for `hf`/`hf_transfer`) and never `pgrep -a` a fetch command that
|
||||
carries an auth header.
|
||||
|
||||
## External Exposure Policy (MANDATORY)
|
||||
|
||||
Every Caddy-proxied service is either behind the auth chain (Anubis :8923 →
|
||||
Authentik embedded outpost) or explicitly internal-only — no public DNS name,
|
||||
no vhost. New services default to internal-only until guarded.
|
||||
|
||||
When guarding a new vhost — provider first, then route:
|
||||
|
||||
1. Create the outpost proxy provider FIRST (`external_host` = exact vhost URL
|
||||
with trailing slash; `internal_host = http://caddy:8017`). If the vhost
|
||||
flips first, authenticated traffic hits the outpost error page.
|
||||
2. Flip the vhost to `reverse_proxy anubis:8923` with
|
||||
`header_up X-Real-Ip {remote_host}`, and add the post-auth
|
||||
`@<svc> host <vhost>` handler on the `http://:8017` listener routing to
|
||||
the real upstream.
|
||||
3. Anubis guard set, Caddy TLS edge, and outpost provider list change
|
||||
together in one deploy.
|
||||
4. Service-to-service traffic targets internal addresses (`loki:3100`), never
|
||||
the public hostname — non-browser clients cannot solve challenges.
|
||||
|
||||
Exempt categories (document the reason in the vhost comment): the IdP itself
|
||||
(`auth.`), git smart-HTTP endpoints, services with their own complete auth.
|
||||
|
||||
## Deletion & Trash Policy (MANDATORY)
|
||||
|
||||
Never `rm`. Move deletions into trash so they stay recoverable:
|
||||
|
||||
1. In-repo files → `.AGENTTRASH/` beside the item's directory.
|
||||
2. Elsewhere → dated staging dir `~/.AGENTTRASH/YYYYMMDD-topic`.
|
||||
3. List staged items and sizes; wait for explicit user approval to purge.
|
||||
|
||||
`rm`, `rm -rf`, and `git clean -f` are forbidden without explicit permission.
|
||||
`git * --no-verify` is forbidden. The trash is a temporary hold, not an
|
||||
archive.
|
||||
|
||||
## Model Swaps (fleet law, 2026-10-18)
|
||||
|
||||
**Swap the model. Do not instrument it.** A payload change is: edit the trio
|
||||
(env / serve script / unit) → `scripts/deploy.sh <host> --service <svc>` →
|
||||
`/health` answers → one chat turn, one tool call, and one vision probe on an
|
||||
endpoint that serves images. That is the entire procedure.
|
||||
|
||||
- **No benchmarking unless Frank orders it.** No pre-swap baseline battery,
|
||||
no A/B arms, no decode/prefill ladders, no acceptance tables, no numbers
|
||||
written into any page afterwards. Data exists only when he asks for it,
|
||||
and then only in the file he names.
|
||||
- **It doesn't work → roll back.** Revert the commit, redeploy, done. No
|
||||
post-mortem page, no investigation log, no trial-drop-in forensic tour. If
|
||||
the fault is not obvious, one line in the endpoint page's *Laws* section is
|
||||
the whole record.
|
||||
- What still gates a change: `./tests/run.sh` before commit, the deployer's
|
||||
per-service health gate, and the Playwright e2e gate. Those are the fleet's
|
||||
evidence. A bench is not evidence of correctness and buys nothing a swap
|
||||
needs.
|
||||
|
||||
### When a bench IS ordered
|
||||
|
||||
**STOP the production service first** — these servers serve real users, and a
|
||||
bench beside a live service starves the GPU and corrupts the result. That
|
||||
covers **any GPU experiment, not just benchmarks**, including a bare
|
||||
2-rank RCCL/`torch.distributed` probe: on 2026-09-16 one was run against a
|
||||
live service "because it only needs 200 MB of VRAM" and the host went hard
|
||||
down ~20 minutes later (no ARP reply, 100 % ICMP loss, power cycle required;
|
||||
a hung AQL dispatch wedging the driver is the plausible mechanism). Telling
|
||||
an off machine from a wedged one: `ip neigh show <ip>` from a peer — FAILED
|
||||
means off or NIC dead; a merely wedged kernel usually still answers ARP.
|
||||
|
||||
1. Stop and mask (self-healing would restart it):
|
||||
`sudo systemctl stop frank-dragoon frank-zealot` on Dankclaw,
|
||||
`sudo systemctl stop frank-archon` on Dankserver, then `mask` each.
|
||||
`mask` FAILS on a relinked unit (the file in `/etc/systemd/system/` is
|
||||
itself a symlink into the checkout); stop alone is enough for the window,
|
||||
and the deploy restores the unit — re-deploy after benching rather than
|
||||
unmasking.
|
||||
2. Bench against the service itself. No scratch ports on an LLM host — one
|
||||
process owns the card. Use the runtime's own harness (`llama-bench`; a
|
||||
live llama.cpp server also reports honest `timings` per request, including
|
||||
`draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build
|
||||
10777 that object only reaches you on **`/v1/chat/completions`**: a greedy
|
||||
`ignore_eos` run on the raw `/completion` path there answers **500 "The
|
||||
model produced output that does not match the expected Content-only
|
||||
format"** (measured 2026-10-18). halogen has no `/completion` path at all,
|
||||
so its rates are wall-clock around `/v1/chat/completions` plus the boot
|
||||
ledger. Generate ≥128 tokens: at 8, `inter_token` is first-token noise and
|
||||
run-to-run spread swamps every real difference.
|
||||
3. Unmask, start, verify `curl http://192.168.0.<host>:7770/health`.
|
||||
Retire every bench drop-in the same session
|
||||
(`systemctl show <svc> -p DropInPaths` must come back empty) and write the
|
||||
numbers into the file Frank named — `/tmp` and the journal are not records.
|
||||
|
||||
Instrument traps that survive from the era when this was routine:
|
||||
`gufo bench` cannot rank speculative decode and **inverts the ranking** (its
|
||||
corpus reports `acceptance=0.933 ± 0.000` with an identical trace sha256
|
||||
across draft widths — the same predictable text every run); llama.cpp's
|
||||
`Server-Timing` carries the unit in the **name** half (`ttft;dur=402.485`),
|
||||
so splitting on `=` keys the dict `ttft;dur` and every lookup silently
|
||||
returns nothing; halogen 0.16.2 answers `/completion`, `/ping`, `/props` and
|
||||
`/tokenize` with 404 and sends no `Server-Timing`; and the first request
|
||||
after a boot is a warmup artifact (measured 2.6 t/s prefill then 1,258 t/s on
|
||||
the next), so never report request #1.
|
||||
|
||||
**CRITICAL:** never `systemctl disable` a system unit on these hosts — it
|
||||
deletes the unit file from `/etc/systemd/system/`, not just the symlink.
|
||||
`mask` symlinks to `/dev/null` and is safe/reversible.
|
||||
|
||||
## Documentation Policy (fleet law, 2026-10-19)
|
||||
|
||||
**Documentation exists for point of use.** A page earns its bytes only if it
|
||||
helps whoever performs the next task — the shape to deploy, the flag that must
|
||||
be passed, the failure signature to recognise, the rollback to run. It is not a
|
||||
lab notebook and not a ledger. **Losing information is an acceptable, usually
|
||||
correct outcome.** Never let an orphan census ("this is the only copy of that
|
||||
fact") decide what to keep: sole custody of a useless fact is still useless.
|
||||
|
||||
- **The purpose test — before you write, and again before you keep:** would the
|
||||
reader act differently because of this line? No → delete it.
|
||||
- **Measurements are disposable.** A rate or throughput figure (t/s, tok/s, ms,
|
||||
acceptance %, a bench ladder, a measured MiB) for a payload, runtime, or
|
||||
layout that is not running now and not being compared now is destroyed, not
|
||||
filed — not in an Eras row, not on a technique page, not "preserved for
|
||||
history". AI moves faster than this vault: the number is wrong before anyone
|
||||
reads it, and a wrong number in a runbook gets believed. Keep the **law** the
|
||||
measurement taught; drop the measurement.
|
||||
- **One fact, one owner**, every other file links with `[[Page]]`. The env file
|
||||
beside the serve script is the authority for every live number; the endpoint
|
||||
page is the authority for the shape and the wire contract.
|
||||
- **One page per endpoint:** current shape, **Eras** (`window | payload |
|
||||
runtime | replaced because | rollback` — the rollback is a commit, not a
|
||||
number), the **Laws**, the wire contract, the files that own it. A swap edits
|
||||
the shape block and appends one row; the retired era's own page is
|
||||
**destroyed** to `.AGENTTRASH/`, never merged into anything.
|
||||
- **`log.md` is retired** (2026-10-19: 2,380 lines destroyed). Git holds the
|
||||
chronology. A causal law belongs to the page that owns the thing it governs,
|
||||
or to *Key Lessons Learned* below if it is cross-cutting — never to a diary.
|
||||
- **Index pages are locators:** one row, one clause, no numbers.
|
||||
|
||||
## Model Config (harness)
|
||||
|
||||
The harness reaches the fleet through `.omp/agent/models.yml` in the
|
||||
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
||||
where the harness runs; there is no copy on the servers. Each provider is one
|
||||
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
||||
the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
|
||||
since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
|
||||
`192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
|
||||
`192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
|
||||
`192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
|
||||
scripts, env files, containers, deploy services and runbooks to match —
|
||||
the three former service names are fully retired and exist nowhere in
|
||||
the repo; historical text in runbooks uses the current names. Verify a
|
||||
rename on `/v1/models`, never
|
||||
on a chat response: radiance echoes the **requested** model name, so a 200
|
||||
proves nothing about the served id (llama.cpp and halogen ignore it).
|
||||
|
||||
Resolution rules: the default model resolves as a `defaultProvider` +
|
||||
`defaultModel` pair (the provider must serve the model); a subagent override
|
||||
uses a bare model `id`, which resolves only when unique across providers. Two
|
||||
ports on one host need two providers — the file cannot express "same host,
|
||||
different port".
|
||||
|
||||
When renaming a service, retiring an endpoint, or moving a model: update the
|
||||
provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
|
||||
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
|
||||
a role pointing at a provider the file does not define fails at call time. As
|
||||
of 2026-10-06 **all three endpoints accept images** (`input: [text,
|
||||
image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
|
||||
(the v2 vision tower loads beside the checkpoint; the text-only halogen
|
||||
27B window closed 2026-10-06) — while
|
||||
`frank-archon` takes them natively (radiance runs the checkpoint's
|
||||
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
||||
are the engine core; `--mm-max-patches` available),
|
||||
and `frank-zealot` through
|
||||
the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
|
||||
publisher's requirement for concurrent image requests (the card states the
|
||||
BF16 tower can crash `llama-server` when several image requests run at once,
|
||||
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
|
||||
**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
|
||||
2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
|
||||
deliberately below the model's native window; llama.cpp answers a longer prompt
|
||||
with a 400 naming the limit, never a truncation, so the harness must compact
|
||||
before it.
|
||||
Note also
|
||||
that
|
||||
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
|
||||
parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
|
||||
models.yml sets `reasoningContentField` per endpoint to match.
|
||||
There is **no audio**
|
||||
input anywhere in the fleet: that capability left with the FLM NPU
|
||||
endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||
|
||||
Each endpoint has ONE page in the vault: shape, Eras, Laws, wire contract,
|
||||
rollback. The numbers below are only what a consumer or a deploy gate needs;
|
||||
the env file beside the serve script is the authority, and a page that
|
||||
disagrees with it is the bug.
|
||||
|
||||
### frank-dragoon — halogen flash-server, Dankclaw iGPU, :7770
|
||||
|
||||
- Owns: `scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
|
||||
`scripts/frank-dragoon.service`, container `frank-dragoon-halogen`, image
|
||||
digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`.
|
||||
Engine stdout (the boot ledger: weights, KV pool, clamp) goes to
|
||||
`/home/dank/logs/frank_dragoon.log`, NOT the journal.
|
||||
- Shape: `HALOGEN_CTX=200704` is the **per-request ceiling** (a longer prompt
|
||||
is a 400, never a truncation); `HALOGEN_KV_SLOTS=2` over ONE
|
||||
`HALOGEN_KV_POOL_POSITIONS=401408` pool — pool positions buy concurrency,
|
||||
the per-request window never moves; `HALOGEN_SPEC_ADAPT=0` keeps the MTP
|
||||
drafter always armed (the adaptive default disarms it and halves decode);
|
||||
server sampler 1.0/0.95/20.
|
||||
- Harness contract: `models.yml` `contextWindow: 155648` = ceiling − 45,056.
|
||||
The clamp is `min(45056, window − est − 4096)` and the chars/4 estimate
|
||||
under-counts, so a harness window equal to the ceiling makes every long
|
||||
request a guaranteed 400. Reasoning arrives in `reasoning_content`; the
|
||||
template rejects `reasoning_effort` outside `xhigh|medium|low`; images are
|
||||
OpenAI `image_url` parts (probe ≥256 px); `response_format` unsupported;
|
||||
`/completion`, `/ping`, `/props`, `/tokenize` are 404.
|
||||
- Page: [[frank-dragoon]].
|
||||
|
||||
### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
|
||||
|
||||
- Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
|
||||
`scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY
|
||||
loader (the payload carries the fork's custom PQ2_0 tensor type plus
|
||||
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
|
||||
loader ignoring that metadata dequantizes wrong); payload
|
||||
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
|
||||
tower can crash the server with concurrent image requests) + the rank-1
|
||||
refusal-direction adapter applied at load by `--lora-scaled …:1`. The
|
||||
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
|
||||
— and build 10777 is also the gate that lets it load: older forks refuse it
|
||||
with `unsupported prism.hadamard.version: 2`.
|
||||
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
|
||||
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
|
||||
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
|
||||
`SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
|
||||
- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
|
||||
(`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
|
||||
with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
|
||||
reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
|
||||
metadata key, so it must reach argv. This endpoint is the fleet's
|
||||
compaction hub.
|
||||
- Page: [[frank-zealot]].
|
||||
|
||||
### frank-archon — radiance, two R9700s TP=2, Dankserver :7770
|
||||
|
||||
- Owns: `scripts/frank-archon.env`, `scripts/serve-frank-archon.sh`,
|
||||
`scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
|
||||
`scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
|
||||
`stilldeadcode/radiance@sha256:d590e300…` (radiance 1.8.1); container name = unit name.
|
||||
- Shape: TP=2 over `wht6`, `--max-num-seqs 2`, `--max-model-len 200000`,
|
||||
`expert_tiered` placement, 44,928 MiB pinned host pool per rank, `fp8` KV,
|
||||
MTP depth 2, MNBT 2048, `--busy-prefill-chunk auto`, prefix cache 8 GiB host
|
||||
tier only, n-gram table pinned to RAM. The per-card weights/KV split is derived
|
||||
by the engine's budget resolver from `--gpu-headroom-mib`, the cache bound and
|
||||
the host pool; the `--vram-weights-mib`/`--vram-kv-mib` pair and the prefix disk
|
||||
tier are gone (removed at 1.3.0 / 1.8.1) and naming either is a boot refusal,
|
||||
not a no-op.
|
||||
- Harness/gate contract: `models.yml` `contextWindow` MUST quote 200,000
|
||||
(over-length is refused, never truncated); the ~147 GiB pinned-RAM ledger
|
||||
is why this is a root unit, and `ttm-preflight.sh` raises
|
||||
`ttm.pages_limit` itself because the kernel resets that knob to half of RAM
|
||||
on every boot; `/health` binds only after the full load, so a bare health
|
||||
gate is honest.
|
||||
- Page: [[frank-archon]].
|
||||
|
||||
## Key Lessons Learned
|
||||
Durable operational lessons from the fleet's history. Full detail in the
|
||||
linked runbooks.
|
||||
|
||||
**PCIe / multi-GPU**
|
||||
|
||||
- **RCCL collectives need PCIe atomics on every participating GPU, and the failure never names
|
||||
atomics.** The kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch them on
|
||||
a device without atomics and the error surfaces as `hipErrorIllegalState` from `hipLaunchKernel`,
|
||||
not as anything that names atomics. The value is consumed at driver PROBE time, so writing the
|
||||
AtomicOp Requester bit with `setpci` at runtime changes nothing. Check the CHAIN: `lspci -vv -s
|
||||
<gpu>` and every upstream bridge must show `AtomicOpsCap: Routing+` (a `Routing-` bridge is a
|
||||
hardware wall), and the endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
|
||||
6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
|
||||
chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
|
||||
nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
|
||||
- **When a GPU collective dies, read the runtime, not the framework.** The message that identified
|
||||
the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
|
||||
`AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
|
||||
failing HIP call with its return code. [[PCIe Link and Atomics]]
|
||||
|
||||
|
||||
**ROCm / GPU**
|
||||
|
||||
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a trap.** With a missing
|
||||
`DST`, cp creates `DST` as a copy of `SRC` and drops one directory level. rocBLAS and hipBLASLt
|
||||
look for their device code at a FIXED depth (`../lib/rocblas/library`,
|
||||
`../lib/hipblaslt/library/<arch>`), so the flattened copy fails at REQUEST time, not load time:
|
||||
the engine reads the weights, answers `/health` and dies on the first completion with
|
||||
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch : gfx1201`. Declare
|
||||
the destination, then copy the contents.
|
||||
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
|
||||
anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
|
||||
and pinning the PLE tensor with `-ot` was slower than letting it page. [[Llama Cpp Serving Sizing]]
|
||||
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
|
||||
with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
|
||||
`1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
|
||||
tunable these; check with
|
||||
`strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
|
||||
- **`GGML_CUDA_FA_ALL_QUANTS` is dead; do not build for it.** Verified on this host 2026-10-19:
|
||||
master's `ggml/cmake/common.cmake` warns it is deprecated in favour of `GGML_CUDA_FA_QUANTS`, the
|
||||
successor's DEFAULT pair list (`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16`) already compiles the
|
||||
quantized-KV kernels with the old flag OFF, and neither llama tree on this host references the old
|
||||
macro anywhere under `ggml/src/ggml-cuda/`. The old "~12× prefill collapse" belongs to the older
|
||||
source line. Prove a build's KV ability from its `CMakeCache.txt`. [[frank-zealot]]
|
||||
- **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
|
||||
`blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
|
||||
loader support.
|
||||
- **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
|
||||
Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
|
||||
`/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
|
||||
`free`/`dmidecode` show the CPU-side carve-out — intentional, do not change. [[Dankclaw System Specs]]
|
||||
|
||||
**Load / crash diagnosis**
|
||||
|
||||
- Model load is slow (minutes cold) and health returns 503 while it runs: do NOT kill the process,
|
||||
`deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On halogen the FIRST request after a (re)start
|
||||
can go silent for ~45–60 s (`engine_unresponsive`, 503s) and then self-recovers — do not restart
|
||||
over it. [[frank-dragoon]]
|
||||
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` AND the kernel line via `journalctl -k` (a
|
||||
segfault at address 8 in `libamdhip64` is a NULL unref in HIP); (2) run the serve script, then the
|
||||
raw binary — a different outcome isolates the env file; (3) `gdb -batch -ex run -ex "bt 25"
|
||||
--args bash scripts/serve-<svc>.sh`, because llama-server buffers stdout and loses the log tail on
|
||||
SIGSEGV: trust gdb, not the log; (4) `amdgpu: SVM mapping failed` spam is a symptom, not a root
|
||||
cause. [[LLM Performance Diagnosis]]
|
||||
- Journal signatures: a client abort logs `W srv stop: cancel task, id_task = N` while a natural stop
|
||||
logs only `slot release` with `truncated = 0`. The server never preempts a running generation;
|
||||
excess tasks queue when slots are busy.
|
||||
|
||||
**Host memory / swap**
|
||||
|
||||
- **Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` of the old one**, or
|
||||
live pages must land in RAM that `MemAvailable` overstates. Dankclaw's 64 GiB `/swap.img`
|
||||
(fstab-driven) protects the co-tenants, not an endpoint with pinned weights — paging the KV pool
|
||||
is worse than the OOM it prevents. Host swap state is invisible to `deploy.sh` (no repo file):
|
||||
`swapon --show` before reasoning about headroom. [[Swap Configuration]]
|
||||
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults to half of RAM, so
|
||||
a manual raise vanished with a power failure, frank-archon's pinned-ledger preflight refused the
|
||||
start, and `Restart=always` re-ran that refusal 6,919 times in ~20 h — **a unit in
|
||||
`activating (auto-restart)` that exits 1 in 50 ms with no container and no engine log is a
|
||||
PREFLIGHT refusal, not an engine fault.** Repo law now: `scripts/ttm-preflight.sh` raises the knob
|
||||
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only after a write the
|
||||
kernel rejected. Never prescribe a manual sysfs write as the recovery procedure for a service that
|
||||
can boot before a human reaches it. [[frank-archon]]
|
||||
- **Silent RAM corruption outranks every software theory.** frank-archon's long-conversation
|
||||
repetition/garbage was marginal DDR5 at 4800 MT/s on four DIMMs, fixed by dropping to 4600.
|
||||
Consumer non-ECC raises NO MCE for a silent flip, and a short in-band pattern check (60 s over
|
||||
8 GiB of 246 GiB) is not a stability proof — both instruments said "ruled out" while the RAM was
|
||||
the cause, and "4800 is the JEDEC floor" is a rating, not a margin guarantee. When the fault is
|
||||
data-shaped (NaN states, garbage tokens) and a software revert does not move it, price in the RAM
|
||||
before the runtime; a revert tested under the same marginal hardware is itself confounded.
|
||||
[[frank-archon]]
|
||||
|
||||
**Runtime-specific guards**
|
||||
|
||||
- **Rootless podman cannot RAISE rlimits, so `--ulimit nofile=1048576` dies at container CREATE**,
|
||||
not at runtime: runc reports `error setting rlimit type 7: operation not permitted` (type 7 =
|
||||
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's ceiling (524288 on
|
||||
Dankclaw). Inherited-equal limits are fine, so `memlock=-1:-1` works iff the unit grants
|
||||
`LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
|
||||
any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
|
||||
before adding flags. [[frank-dragoon]]
|
||||
- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a
|
||||
file to a loader.** A non-standard tensor type in the KV section makes stock loaders refuse
|
||||
the file at load, so a fork can be the only door rather than a preference. Counting tensor
|
||||
types takes one python read and settles a runtime argument in minutes; build identity matters
|
||||
as much as the fork, so pin the commit by SHA in any build script. [[GGUF Compatibility]]
|
||||
- **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
|
||||
(the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
|
||||
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
|
||||
measures a crash instead of the absence of drafting. [[frank-zealot]]
|
||||
- **An orphaned scratch server taxes the whole host.** A bench child left alive by a client-side
|
||||
exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
|
||||
every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
|
||||
back before believing a number — and kill by pid, never `pkill -f llama-server`.
|
||||
|
||||
**llama.cpp serving**
|
||||
|
||||
- **MTP is a property of the FILE, not of the flags.** The `-mtp` export carries the head as
|
||||
`blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`); `--spec-type draft-mtp` sets
|
||||
`load_mtp` and the draft context runs on the MAIN model's weights (`shares_model`), so there
|
||||
is no second copy of a 12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
||||
tensors at all, and pointing `--model-draft` at that file loads it twice. [[frank-zealot]]
|
||||
- **What can be chained: ONE context-backed drafter plus any number of draftless ngram types.**
|
||||
`--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
|
||||
`ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
|
||||
`draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
|
||||
legal and it wins on prompt-echo traffic, but `draft-mtp,draft-dflash`
|
||||
dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
|
||||
chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
|
||||
hybrid loses, because adding the ngram layer logs
|
||||
`recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
|
||||
MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
|
||||
[[frank-dragoon]]
|
||||
- **A drafter's KV scales with the TARGET's context, and its window policy decides the price.**
|
||||
The draft context is built at `llama_n_ctx(ctx_tgt)` with no flag to cap it, so a drafter
|
||||
declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
|
||||
first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
|
||||
drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
|
||||
compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
|
||||
wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
|
||||
cost against what the batch knobs leave for compute. Measure the arm, never the flag.
|
||||
[[frank-zealot]]
|
||||
- **A quant's scale overhead belongs in its byte rate; doubling a measured figure to "get the
|
||||
f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element (32 codes + one 2-byte scale per
|
||||
32-block), f16 is exactly 2.0 ⇒ the ratio is **1.882×**, not 2×. Derive byte rates from element
|
||||
count × block layout, then check them against the boot ledger — this repo carried an invented
|
||||
2× number for weeks. Current KV math: [[frank-zealot]]
|
||||
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is BACKEND-SPECIFIC — never
|
||||
carry it across hosts.** Its compute buffers land about TWICE as heavy on the card that holds
|
||||
the output layers and the drafter, so a split balanced at one batch is unbalanced at another:
|
||||
re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
|
||||
large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
|
||||
model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
|
||||
control that proves the knob, not the traffic, moved the number. The optimum is a property of
|
||||
the backend, so re-find it after any device or payload change. When a bench IS ordered, A/B it
|
||||
by restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
|
||||
port: one card holds one server.
|
||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
|
||||
boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
|
||||
by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
|
||||
matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
|
||||
only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
|
||||
costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
|
||||
too short and it proposes on ordinary prose fragments, costing verification without acceptance.
|
||||
[[Llama Cpp Serving Sizing]]
|
||||
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
|
||||
the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
|
||||
prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
|
||||
`contextWindow` MUST follow every reslice. Both slot directions cost or buy window, never
|
||||
pool; shorter slots decode slightly faster. Measured reslices: [[frank-zealot]].
|
||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj costs ~1.5× its file
|
||||
size** (the encoder's activation and compute buffers ride with it). Subtracting two booted rows
|
||||
isolates it exactly; extrapolating weights+KV+"a bit for buffers" is what leaves 1.1 GiB free
|
||||
when you meant 2.7. Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||
`PARALLEL × 4096`. **The projector is NOT billed in llama.cpp's own fit check**: the engine
|
||||
prints its worst case and fits WITHOUT it, so price a shape from settled `nvidia-smi` after a
|
||||
real boot — the projector, and any drafter context, lands after `/health` would already have
|
||||
answered (same class as the DSpark drafter OOM). Two-boot table: [[frank-zealot]].
|
||||
- **`images` and `image_url` are NOT interchangeable: only the content part reaches the model.**
|
||||
`--mmproj` loads, `usage.prompt_tokens` rises, and the model still invents an answer if the
|
||||
client used the legacy top-level `images` field on `/v1/chat/completions` — frank-zealot
|
||||
answered "Brown." for a solid-red PNG that way and "Red." through the OpenAI `image_url` part.
|
||||
The failure is silent because a hallucinated color is a plausible answer; probe with a solid
|
||||
color and expect an exact word. [[frank-zealot]]
|
||||
- **`--jinja` is what `tools` needs.** A subagent died with `500 tools param requires --jinja
|
||||
flag` on the ik_llama.cpp runtime that served frank-dragoon: every tool-using agent on that
|
||||
endpoint was broken and the failure read like a model fault. Mainline llama.cpp defaults
|
||||
`--jinja` on and the serve scripts pin it; halogen also accepts `tools` and returns
|
||||
`tool_calls`. Check tool support before routing a role at an endpoint.
|
||||
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's template accepts
|
||||
only `xhigh`, `medium` and `low` for `reasoning_effort` and raises on anything else — HTTP 500,
|
||||
not a clamp — so the harness's levels must be folded onto that set in `models.yml`
|
||||
(`compat.reasoningEffortMap`). A role whose level is unmapped fails on its FIRST request and
|
||||
the failure names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||
`Unexpected reasoning effort high` until `high: xhigh` was added. Halogen accepts the wider
|
||||
set, so the two entries legitimately differ.
|
||||
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe reads as
|
||||
blindness.** A probe at `max_tokens: 64` returned `content: ""` with `finish_reason: "length"`
|
||||
and half a thought in `reasoning_content` while the image HAD reached the model (322 prompt
|
||||
tokens vs 42 for the text alone). Budget ≥512 for any thinking model you probe for vision.
|
||||
- **The FIRST request after a boot is a warmup artifact**, and the 3× spread on one shape makes
|
||||
graph-capture cost read as a model fault. Benchmark the second request or later.
|
||||
[[frank-zealot]]
|
||||
- **A fork build hides its own dtype laws.** Vendor builds carry their own compile options, so
|
||||
prove a build's KV behaviour from the KV buffer line in the boot ledger and its
|
||||
`CMakeCache.txt` — never from the build flags you remember passing. [[frank-zealot]]
|
||||
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed**: without it
|
||||
the engine answers **501 Not Implemented** on `/metrics`, so the scrape fails while `/health`
|
||||
answers 200 and `LlmEndpointDown` fires beside a healthy service. halogen serves `/metrics`
|
||||
natively, so the asymmetry reads as "only the 3090 is broken". Diagnose with `lastError` from
|
||||
`/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot
|
||||
re-stamps `startsAt` on an alert that has fired for weeks. [[Observability]] [[frank-zealot]]
|
||||
- **A leftover bench drop-in silently hijacks the next deploy, and no health gate sees it.** The
|
||||
sanctioned bench-window method (a transient `Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`
|
||||
drop-in sourcing the repo env) outlives the bench unless retired: a leftover
|
||||
`frank-zealot.service.d/ptq1.conf` made the cutover deploy boot a **different model at a
|
||||
different shape** while `/health` answered 200, `/v1/models` echoed the alias (a flag, not a
|
||||
fact) and the Playwright gate passed 33/33. After deploying any recently-benched LLM service:
|
||||
`systemctl show <svc> -p DropInPaths` (must be empty), then
|
||||
`tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger — and retire drop-ins
|
||||
to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`. [[frank-zealot]]
|
||||
- **A bench verdict that never reaches the repo is indistinguishable from a rogue arm.** That
|
||||
same arm served production for 24 minutes with real numbers (~48 t/s decode, 801 t/s prefill at
|
||||
141k) that lived only in `/tmp` env files and journald, so the next agent read the surviving
|
||||
drop-in as an unfinished experiment and reported a hijack. Write the measurement into the
|
||||
endpoint page and the arm script's header in the session that takes it — `/tmp` and the journal
|
||||
are not records.
|
||||
- **`llama-bench` separates a LIST with `/` and SWEEP values with `,`** — for both `-dev` and
|
||||
`-ts`. `-dev CUDA0,Vulkan1` is two *single-device* configurations, never one two-device run,
|
||||
and `-ts 0.85,0.15` is two sweep values each putting everything on device 0. Write
|
||||
`-dev A/B -ts x/y`. This artifact produced a wrong fleet conclusion ("CUDA + Vulkan cannot serve
|
||||
one model") from two bogus OOMs, and the bench table's own `dev` column was printing the truth
|
||||
the whole time. Any placement-dependent number needs an independent check: sample the other
|
||||
device's memory (`nvidia-smi`) while the run is loaded.
|
||||
|
||||
**Ops**
|
||||
|
||||
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
|
||||
Use `pkill -x <exact-name>`.
|
||||
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
|
||||
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
|
||||
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
|
||||
`GroupAdd=keep-groups`, plus the udev rule that makes those nodes world-writable — user-unit
|
||||
spawns carry no supplementary groups) and must NOT pass `--cpu`: the gfx1151 ROCm wheels exist
|
||||
and a stale device-less container was found restarted 9,146 times. [[ComfyUI]]
|
||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun
|
||||
service. [[SearXNG]] [[Gluetun]]
|
||||
- **Dankclaw's kernel command line is a repo file.** It boots **`iommu=pt`** since 2026-10-18,
|
||||
tracked in `dankclaw/host/grub`, and moves ONLY through `scripts/set-dankclaw-bootline.sh`
|
||||
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by hand-editing
|
||||
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
|
||||
says identity mode costs nothing, so the XDNA2 NPU returned for free.
|
||||
[[dankclaw-iommu-translation-off]]
|
||||
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
|
||||
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
|
||||
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
|
||||
[[Quadlet Lifecycle]]
|
||||
- **A dead rootless podman pause process breaks ALL container management on the host, and
|
||||
`podman system migrate` panics in 5.7.0.** Every `podman` command fails with
|
||||
`invalid internal status`, every start fails binding the bridges
|
||||
(`failed to bind udp listener on 10.89.x.1:53`), every `Restart=always` quadlet loops — while
|
||||
container storage stays intact. What is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
|
||||
token) and the rootless netns; deleting the stale token is not enough. Repair: **reboot**
|
||||
(`/run/user` clears and the quadlets return on their own — check `Linger=yes` and
|
||||
`WantedBy=default.target` first). The 2026-09-16 occurrence had no OOM in `journalctl -k`:
|
||||
treat the pause process as host state that can die for its own reasons. [[Quadlet Lifecycle]]
|
||||
- **A container cannot reach a `Network=host` service at the host's LAN IP**: netavark DNATs
|
||||
*published* ports only, so the host bind and the `10.89.1.1` gateway are both refused from a
|
||||
bridge container while the published port answers. Use `host.containers.internal`
|
||||
(169.254.1.2). This 502'd `hermes.franksdank.cloud` until its Caddyfile upstream changed.
|
||||
[[Container Host Reachability]]
|
||||
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as `.partNofM`
|
||||
fragments** while the smaller rungs are single files, so filtering `endswith(".gguf")` makes a
|
||||
whole quant class look absent. Fetch the parts, verify each against the Hub's LFS sha256,
|
||||
concatenate in numeric order. [[Model Swap and Rollback]]
|
||||
- **A fetch script must `-f` and verify, or it reports success on an access error.**
|
||||
`curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
|
||||
repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
|
||||
seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
|
||||
declared size, and check the LFS sha256 — the live `scripts/download-frank-*.sh` scripts do
|
||||
all three; ad-hoc fetches tend not to. Gated repos 403 even with the host's HF token until
|
||||
that account accepts the gate. [[Model Swap and Rollback]]
|
||||
- **A verdict is a lead — the scout's and the tool's alike.** In the 2026-10-19
|
||||
documentation campaign six scouts worked the same pages and produced invented
|
||||
bullet IDs, files that do not exist, "duplications" grep disproved, line
|
||||
numbers past the end of the file they judged, and Eras tables with invented
|
||||
payloads. The mechanical tool was worse: `scripts/port-census.py` built its
|
||||
corpus from the directory containing the page under test, so each page was held
|
||||
by *itself* and ten era pages reported "0 sole-record lines" — which reads as
|
||||
"pure duplication, trash it". Nothing was cut on a verdict; every holder was
|
||||
re-grepped and every tool re-measured before it was believed.
|
||||
- **Never let a census decide what to keep.** That same campaign then concluded
|
||||
the era pages were irreplaceable because each held 14–63 sole-record lines —
|
||||
wrong in the opposite direction: those were throughput figures for payloads that
|
||||
no longer run. Point of use is the only ownership test, the operator applies it,
|
||||
and 53 pages plus the 2,380-line change diary were destroyed as a result. A tool
|
||||
can say where a fact lives; it cannot say whether anything needs it.
|
||||
|
||||
## Runbook Index
|
||||
|
||||
One page per live endpoint; every payload swap it has ever run is a row in that
|
||||
page's **Eras** table, with the rollback path in the same row. There is no
|
||||
per-attempt page and no per-swap ledger anywhere else.
|
||||
|
||||
| Service | Page |
|
||||
|---------|------|
|
||||
| `frank-archon` — radiance, 2× R9700 TP=2, Dankserver | `serverAppConfigs-docs/LLM/frank-archon.md` |
|
||||
| `frank-dragoon` — halogen flash-server, Dankclaw iGPU | `serverAppConfigs-docs/LLM/frank-dragoon.md` |
|
||||
| `frank-zealot` — llama.cpp (PrismML fork), RTX 3090 | `serverAppConfigs-docs/LLM/frank-zealot.md` |
|
||||
| `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
|
||||
| Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
|
||||
|
||||
There is no page for a retired endpoint and no change diary. A payload swap
|
||||
appends one row to the live endpoint's **Eras** table; when the endpoint itself
|
||||
dies, its page is destroyed. Rollback is a commit, not a document.
|
||||
`serverAppConfigs-docs/index.md` is the locator for every page that survives;
|
||||
retired scripts live in `old/` and `.AGENTTRASH/`.
|
||||
|
||||
## Agent skills
|
||||
|
||||
### Issue tracker
|
||||
|
||||
Issues live in Gitea (`dank/serverAppConfigs` on Dankclaw), via its REST API with the stored credential. See `serverAppConfigs-docs/Agent/Issue Tracker.md`.
|
||||
|
||||
### Triage labels
|
||||
|
||||
The five canonical roles keep their default strings (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `serverAppConfigs-docs/Agent/Triage Labels.md`.
|
||||
|
||||
### Domain docs
|
||||
|
||||
Single-context: root `GLOSSARY.md` + `docs/adr/`, created lazily; this vault stays the owner of operational facts. See `serverAppConfigs-docs/Agent/Domain Docs.md`.
|
||||
|
||||
## Why
|
||||
|
||||
Direct edits on the remote drift from version control, and future agents
|
||||
won't know what changed. Git is the single source of truth. The two workloads
|
||||
share two hosts, one Caddy, and one Gitea; one repo keeps them in one
|
||||
history.
|
||||
|
||||
## Summary
|
||||
|
||||
| Context | Action |
|
||||
|---------|--------|
|
||||
| Editing a compose/Caddyfile/serve script | **Through git** — fetch, commit, edit, test, push, pull, deploy. Never edit on the remote. |
|
||||
| Config exists only on remote | **Pull it into the repo first** — commit the fetched version (redacted) before editing. |
|
||||
| Unstaged local changes | **Commit or discard** — check `git status` before starting work; ask if unsure. |
|
||||
| Deploying a committed change | **`scripts/deploy.sh <target>`** — pushes, pulls ff-only, plans, dispatches, health-checks. |
|
||||
| Container without a Quadlet unit | **Give it one** — boot-persistent, self-healing `.container` via git. |
|
||||
| Changing a model/payload | **Swap it** — edit the trio, deploy, `health` + one chat/tool/vision probe. No benches unless Frank orders one; if it misbehaves, revert. |
|
||||
| Writing a doc | **One owner per fact** — edit the endpoint/service page; everywhere else links. Never restate a number. |
|
||||
| Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
|
||||
| Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
|
||||
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
|
||||
| Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |
|
||||
### Commit Guidelines
|
||||
- Use descriptive commit messages: `git commit -m "Add AWQ DFlash script"`
|
||||
- Commit before deploying: `git add -A && git commit -m "Update for deployment"`
|
||||
- Push to remote: `git push origin <branch>`
|
||||
|
||||
### Deployment Flow
|
||||
1. Make changes locally
|
||||
2. Run tests: `bash tests/test_validator.sh dankserver`
|
||||
3. Commit: `git add -A && git commit -m "Description"`
|
||||
4. Deploy: `bash deploy.sh dankserver`
|
||||
5. Verify: `bash scripts/validate-vllm.sh -t dankserver`
|
||||
|
||||
## Remote Servers
|
||||
|
||||
### dankserver
|
||||
- SSH: `dank@dankserver`
|
||||
- Scripts: `~/vllm/`
|
||||
- Service: `vllm-awq-qwen36.service`
|
||||
- Model: frank-default-small (AWQ 4bit)
|
||||
- Port: 34401
|
||||
- Status: Disabled (pending bug fixes)
|
||||
|
||||
### dankclaw
|
||||
- SSH: `dank@dankclaw`
|
||||
- Scripts: `~/vllm/`
|
||||
- Service: `vllm-thinkingcap.service`
|
||||
- Model: frank-default-large (FP8)
|
||||
- Port: 3090
|
||||
- Status: Running (you are the deployed inference)
|
||||
|
||||
## Troubleshooting
|
||||
- Service failed: `systemctl status <service>`
|
||||
- Container stuck: `docker stop <name> && docker rm <name>`
|
||||
- OOM error: Reduce `--max-model-len` or increase `--cpu-offload-gb`
|
||||
- Prefix caching conflict: Remove `--enable-prefix-caching` from script
|
||||
|
||||
@@ -1,74 +1,105 @@
|
||||
# Server App Configs
|
||||
# vLLM Scripts
|
||||
|
||||
### AI Disclosures:
|
||||
|
||||
- Slop rating: 2/5 - Actively using, actively de-slopping, **repository mostly human owned**, lacks formal process.
|
||||
- Completeness: Unfinished, planning planned, no plan yet
|
||||
- Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
|
||||
- Local model: Qwen 3.8 Flash Next
|
||||
|
||||
### slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
|
||||
### slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
|
||||
|
||||
|
||||
The single repository for the two homelab servers: the Podman container
|
||||
stacks **and** the LLM inference fleet. One repo, one checkout per host,
|
||||
one deploy script.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
|
||||
> into this repo with its full git history. See AGENTS.md for the merge
|
||||
> note.
|
||||
Scripts for running large language models via vLLM on two local machines.
|
||||
|
||||
## Targets
|
||||
|
||||
| Host | User | IP | Podman stack | LLM service | LLM port |
|
||||
|------|------|----|--------------|-------------|----------|
|
||||
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
|
||||
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
|
||||
| Host | User | Codename | Architecture | Service Status |
|
||||
|------|------|----------|-------------|----------------|
|
||||
| dankserver | dank | DankServer | NVIDIA GPU + System RAM | `vllm-awq-qwen36.service` (failed) |
|
||||
| dankclaw | dank | DankClaw | AMD Strix Halo (Shared Memory) | `vllm-thinkingcap.service` (failed) |
|
||||
|
||||
## Active Services
|
||||
## File Locations
|
||||
|
||||
| Service | Host | Hostname | Port |
|
||||
|---------|------|----------|------|
|
||||
| RomM | dankserver | romm.dankserver.local | 8085 |
|
||||
| Copyparty | dankserver | files.dankserver.local | 3923 |
|
||||
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
|
||||
| frank-medium | dankserver | — (bare endpoint) | 7770 |
|
||||
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
|
||||
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
|
||||
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
|
||||
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
|
||||
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
|
||||
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
|
||||
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
|
||||
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
|
||||
### Local (vllmScripts/)
|
||||
- `scripts/` — Active inference scripts
|
||||
- `tests/` — Test suite
|
||||
- `docs/` — Knowledge base
|
||||
- `failedAttempts/` — Archived scripts
|
||||
|
||||
### Remote (dankserver)
|
||||
- `~/vllm/` — Deployed inference scripts
|
||||
- `~/vllm/failedAttempts/` — Archived scripts
|
||||
- `~/vLLM/` — Legacy project directory (vllm-qwen36.sh)
|
||||
|
||||
### Remote (dankclaw)
|
||||
- `~/vllm/` — vLLM source code (not inference scripts)
|
||||
- `~/vLLM/` — Inference scripts (if deployed)
|
||||
|
||||
## Models
|
||||
|
||||
| Model | Target | Format |
|
||||
|-------|--------|--------|
|
||||
| Qwen3.6 35B A3B FP8 | dankserver | FP8 |
|
||||
| Qwen3.6 27B FP8 | dankclaw | FP8 |
|
||||
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ 4bit |
|
||||
| Qwen3.6 35B A3B DFlash | dankserver | Draft model |
|
||||
|
||||
## Structure
|
||||
|
||||
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the Dankserver stack
|
||||
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the Dankclaw stack
|
||||
- `shared/` — homepage, prometheus config, deploy helpers
|
||||
- `scripts/` — `deploy.sh` and `deploy-remote.sh` (service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnesses
|
||||
- `old/` — retired PP/NPU-era LLM scripts (reference only)
|
||||
- `serverAppConfigs-docs/` — project knowledge base (Obsidian vault, wikilinks). `serverAppConfigs-docs/LLM/` is the LLM fleet
|
||||
- `tests/` — bats test suite (see [[Testing Policy]])
|
||||
|
||||
## Deployment
|
||||
|
||||
See [serverAppConfigs-docs/Architecture/Deployment Flow.md](serverAppConfigs-docs/Architecture/Deployment%20Flow.md)
|
||||
and [serverAppConfigs-docs/LLM/deployment.md](serverAppConfigs-docs/LLM/deployment.md). Run
|
||||
`./scripts/deploy.sh <dankclaw|dankserver>` after committing changes. It
|
||||
plans and deploys only the changed services. Use `--service NAME` for an
|
||||
explicit single-service retry; inference is restarted only when its own files
|
||||
(or the shared chat template) changed.
|
||||
- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, and deployment notes
|
||||
- `scripts/` — Executable vLLM launch scripts
|
||||
- `vllm-qwen36.sh` — FP8 inference script for dankserver
|
||||
- `vllm-awq-qwen36-dflash.sh` — AWQ 4bit with DFlash speculative decoding (docker)
|
||||
- `vllm-awq-qwen36-dflash-ampere.sh` — AWQ 4bit with DFlash speculative decoding (Ampere-optimized, systemd)
|
||||
|
||||
## Knowledge Base
|
||||
|
||||
The `serverAppConfigs-docs/` directory is an Obsidian vault. Start at
|
||||
[serverAppConfigs-docs/index.md](serverAppConfigs-docs/index.md). The LLM fleet knowledge base lives at
|
||||
[serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
|
||||
See [[docs/README.md]] for the project knowledge base index.
|
||||
|
||||
## Testing
|
||||
## Lessons Learned
|
||||
|
||||
Run `./tests/run.sh` before every push. See
|
||||
[serverAppConfigs-docs/Architecture/Testing Policy.md](serverAppConfigs-docs/Architecture/Testing%20Policy.md).
|
||||
### AWQ + DFlash on Ampere Hardware (2026-08-08)
|
||||
|
||||
**What worked:**
|
||||
- `cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit` loads successfully on Ampere GPUs
|
||||
- AWQ Marlin kernel engaged automatically for weight conversion
|
||||
- DFlash draft model `z-lab/Qwen3.6-35B-A3B-DFlash` detected as EAGLE model
|
||||
- torch.compile with cudagraphs works with `--max-num-batched-tokens 1536`
|
||||
|
||||
**What didn't work:**
|
||||
- `--enable-prefix-caching` conflicts with Mamba cache "align" mode — causes `AssertionError` in `unify_kv_cache_spec_page_size`
|
||||
- `--max-model-len 262144` causes OOM during AWQ→Marlin weight conversion on 24GB GPU
|
||||
- Nightly vLLM `--speculative-config` requires `num_speculative_tokens` when model is provided
|
||||
- `--speculative-model` flag removed in nightly; use `--speculative-config` JSON instead
|
||||
|
||||
**Working configuration:**
|
||||
- `--max-model-len 32768` (32k context)
|
||||
- `--cpu-offload-gb 40` with `--cpu-offload-params "experts"`
|
||||
- `--gpu-memory-utilization 0.95`
|
||||
- `--performance-mode throughput`
|
||||
- `--enable-chunked-prefill`
|
||||
- `--max-num-batched-tokens 1536`
|
||||
- Thread binding: `OMP_NUM_THREADS=8`, `VLLM_CPU_OMP_THREADS_BIND=8`
|
||||
|
||||
**Performance observations (no speculative decoding):**
|
||||
| Streams | Prompt Speed | Completion Speed | Total Time |
|
||||
|---------|-------------|-----------------|------------|
|
||||
| 1 | 4.27 tok/s | 21.42 tok/s | 11,951ms |
|
||||
| 2 | 2.44 tok/s | 12.25 tok/s | 20,891ms |
|
||||
| 3 | 1.35 tok/s | 6.76 tok/s | 37,849ms |
|
||||
| 4 | 1.43 tok/s | 7.19 tok/s | 35,606ms |
|
||||
|
||||
Note: Speeds decrease with more streams due to KV cache contention.
|
||||
|
||||
**Still needs work:**
|
||||
- DFlash speculative decoding integration (KV cache assertion bug in nightly)
|
||||
- Full context length support (32k is temporary workaround for OOM)
|
||||
|
||||
## Validation
|
||||
|
||||
Use `scripts/validate-vllm.sh` to test inference servers:
|
||||
|
||||
```bash
|
||||
# Test dankserver (default)
|
||||
bash scripts/validate-vllm.sh -t dankserver
|
||||
|
||||
# Test dankclaw
|
||||
bash scripts/validate-vllm.sh -t dankclaw
|
||||
|
||||
# Custom host/port
|
||||
bash scripts/validate-vllm.sh -H 192.168.1.100 -P 8001 -M my-model
|
||||
|
||||
# Run tests
|
||||
bash tests/test_validator.sh dankserver
|
||||
```
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
|
||||
#
|
||||
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
|
||||
# there, and the benchmark must own those cards, so the whole stop -> boot
|
||||
# scratch -> bench -> restore sequence runs ON dankserver (see
|
||||
# scripts/autoresearch-frank-archon.sh). This wrapper:
|
||||
# 1. copies the candidate config (the repo's scripts/frank-archon.env, the
|
||||
# file the autoresearch loop edits) and the bench script to dankserver,
|
||||
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
|
||||
# exit code back here verbatim.
|
||||
# It deliberately does NOT push to origin: the scp dirties dankserver's
|
||||
# working tree, and the loop's uncommitted candidate config must not be
|
||||
# pushed. Only this file and scripts/autoresearch-frank-archon.sh are
|
||||
# committed; the candidate env rides over ssh each run.
|
||||
set -uo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REMOTE=dankserver
|
||||
REMOTE_REPO=/home/dank/serverAppConfigs
|
||||
|
||||
scp -q "$DIR/scripts/frank-archon.env" \
|
||||
"$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
|
||||
scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
|
||||
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
|
||||
|
||||
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
|
||||
@@ -1,101 +0,0 @@
|
||||
# Dankclaw stack secrets and configuration.
|
||||
# Copy to .env and fill real values. Never commit real secrets.
|
||||
|
||||
# SearXNG
|
||||
SEARXNG_SECRET=changeme
|
||||
SEARXNG_BASE_URL=http://search.franksdank.cloud/
|
||||
# Kagi search API key (https://help.kagi.com/kagi/api/). Powers the paid
|
||||
# `kagi` engine in shared/searxng/settings.yml; deploy-remote.sh bakes it into
|
||||
# the live settings.yml via scripts/searxng-render-settings.py. The quadlet
|
||||
# does NOT pass it into the container environment.
|
||||
KAGI_API_KEY=changeme
|
||||
|
||||
# Gluetun VPN (see https://github.com/qdm12/gluetun-wiki)
|
||||
# Single-peer WireGuard model. Fill from your provider's WireGuard config.
|
||||
VPN_SERVICE_PROVIDER=protonvpn
|
||||
VPN_TYPE=wireguard
|
||||
WIREGUARD_PRIVATE_KEY=changeme
|
||||
WIREGUARD_ADDRESSES=0.0.0.0/32
|
||||
WIREGUARD_PUBLIC_KEY=changeme
|
||||
WIREGUARD_ENDPOINT_IP=0.0.0.0
|
||||
WIREGUARD_ENDPOINT_PORT=51820
|
||||
WIREGUARD_ALLOWED_IPS=0.0.0.0/0,::/0
|
||||
WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=25s
|
||||
|
||||
# Gitea database
|
||||
GITEA_DB_PASSWORD=changeme
|
||||
|
||||
# Grafana OAuth2 client secret (authentik OAuth2Provider "grafana", pk 9).
|
||||
# Created via ak shell on authentik-server; the live .env must hold the
|
||||
# provider's generated client_secret verbatim. Rotation = rotate the provider
|
||||
# secret in authentik, restart grafana after updating the live .env value.
|
||||
GRAFANA_OAUTH_CLIENT_SECRET=changeme
|
||||
|
||||
# Caddy (the homelab's single reverse proxy, on this host)
|
||||
# bcrypt hash (caddy hash-password). deploy.sh renders it into
|
||||
# /home/dank/Containers/caddy/Caddyfile, which the container mounts.
|
||||
# CADDY_BASICAUTH_HASH: the shared family password, now used only by the
|
||||
# pihole vhost (grafana moved behind the Authentik guard chain 2026-09-01).
|
||||
# Username: dankuser.
|
||||
CADDY_BASICAUTH_HASH=changeme
|
||||
# (The homepage's per-user hashes were retired: the homepage and its /admin/
|
||||
# page are now behind Authentik forward_auth, not basicauth.)
|
||||
|
||||
# Hermes -> Signal (signal-cli daemon on the host, port 8080)
|
||||
# E.164 number of the linked Signal account (your primary number, for Note-to-Self).
|
||||
SIGNAL_ACCOUNT=+15551234567
|
||||
# Comma-separated E.164 numbers allowed to message the agent.
|
||||
SIGNAL_ALLOWED_USERS=+15551234567
|
||||
|
||||
# Hermes web UI (Hermex server) login password. Required: the webui binds to
|
||||
# loopback and is reached only through the SSH tunnel, but Hermex still signs
|
||||
# in with this password. Set a strong value in the live .env.
|
||||
HERMES_WEBUI_PASSWORD=changeme
|
||||
|
||||
# Porkbun API (franksdank.cloud + franksdank.casa zones). Caddy uses these
|
||||
# for DNS-01 Let's Encrypt wildcard cert issuance. Porkbun API keys are
|
||||
# account-wide; rotate them here rather than in the dashboard.
|
||||
# PORKBUN_API_SECRET_KEY is the name the caddy-dns/porkbun module expects.
|
||||
PORKBUN_API_KEY=changeme
|
||||
PORKBUN_API_SECRET_KEY=changeme
|
||||
|
||||
# Authentik (SSO / OIDC provider)
|
||||
# AUTHENTIK_PG_PASS: the PostgreSQL password for authentik-postgres.
|
||||
# Generate: openssl rand -base64 32
|
||||
AUTHENTIK_PG_PASS=changeme
|
||||
# AUTHENTIK_SECRET_KEY: the server's secret key (64+ chars).
|
||||
# Generate: openssl rand -base64 48
|
||||
AUTHENTIK_SECRET_KEY=changeme
|
||||
|
||||
# Marreta (content-cleaning proxy, behind the Authentik guard chain).
|
||||
# deploy-remote.sh seeds /home/dank/Containers/marreta/env from these three
|
||||
# values (only if that file does not already exist) and the marreta quadlet
|
||||
# maps them to the app's APP_KEY / ADMIN_EMAIL / ADMIN_PASSWORD.
|
||||
# MARRETA_APP_KEY: Laravel encryption key. Generate:
|
||||
# openssl rand -base64 32
|
||||
# Changing it after first boot invalidates encrypted session/cookie data.
|
||||
MARRETA_APP_KEY=changeme
|
||||
# Admin panel login (marreta.franksdank.cloud/admin). The app creates this
|
||||
# account on first boot from these values.
|
||||
MARRETA_ADMIN_EMAIL=admin@franksdank.cloud
|
||||
MARRETA_ADMIN_PASSWORD=changeme
|
||||
|
||||
# open-notebook (podcast generator; internal-only, no vhost).
|
||||
# deploy-remote.sh seeds /home/dank/Containers/open-notebook/env from these
|
||||
# values (only if that file does not already exist) and refuses to start the
|
||||
# units while any of them is blank: a blank OPEN_NOTEBOOK_PASSWORD means
|
||||
# authentication OFF, and a blank encryption key means no provider credential
|
||||
# can be saved. The seed writes SURREAL_PASS too — the spelling the `surreal`
|
||||
# CLI reads — from SURREAL_PASSWORD, so the app and the database cannot be
|
||||
# given different values.
|
||||
# OPEN_NOTEBOOK_ENCRYPTION_KEY: encrypts the stored AI provider keys at rest.
|
||||
# Generate: openssl rand -hex 32
|
||||
# Changing it after credentials are saved makes them undecryptable.
|
||||
OPEN_NOTEBOOK_ENCRYPTION_KEY=changeme
|
||||
# UI/API password. Every API route — including the episode-audio download the
|
||||
# app serves without a session — is open to the LAN while this is unset.
|
||||
OPEN_NOTEBOOK_PASSWORD=changeme
|
||||
# SurrealDB root password (the root user NAME comes from
|
||||
# scripts/open-notebook.env, so this file holds only the secret).
|
||||
# Generate: openssl rand -base64 32
|
||||
SURREAL_PASSWORD=changeme
|
||||
@@ -1,462 +0,0 @@
|
||||
# Homelab Caddyfile — the single Caddy for both hosts.
|
||||
#
|
||||
# Caddy runs as a container on dankclaw (192.168.0.143, this repo's
|
||||
# compose.yaml service 'caddy') and fronts every service on both hosts:
|
||||
# - dankclaw services: by container name (compose network) or by the
|
||||
# published LAN port (quadlet services on isolated networks)
|
||||
# - dankserver services (192.168.0.235): by published LAN port
|
||||
#
|
||||
# Naming: flat franksdank.cloud scheme (see serverAppConfigs-docs/Infrastructure/LAN-DNS.md).
|
||||
# DNS for these names is served by the Pi-hole quadlet on this host
|
||||
# (dankclaw/pihole/addn-hosts — every name resolves to 192.168.0.143)
|
||||
# and handed to LAN clients by the SURFboard G36 router.
|
||||
#
|
||||
# Authentication chain (2026-08-29): the guarded vhosts (homepage, SearXNG,
|
||||
# Prometheus, RomM, Copyparty) proxy EVERYTHING through the Anubis
|
||||
# bot-challenge proxy first, then the whole request is proxied THROUGH
|
||||
# Authentik (2026.8 proxy-provider mode: this build serves no forward_auth
|
||||
# endpoint; the authentication flow runs inline — unauthenticated clients get
|
||||
# the login UI served in place, and once a session exists Authentik proxies
|
||||
# the request on to the app via the provider's internal_host). Chain: client →
|
||||
# Caddy (TLS, public listener) → Anubis :8923 (verify: DENY known AI scrapers,
|
||||
# CHALLENGE everyone else) → Caddy's internal listener http://:8016 (auth
|
||||
# decision; Host headers survive every hop) → authenticated traffic is
|
||||
# forwarded back to http://:8017 and routed straight to the app.
|
||||
# Apps with their own auth (Gitea, OpenWebUI, Jellyfin, *arr, Pi-hole) and the
|
||||
# Authentik IdP itself (auth.franksdank.cloud — its login flow must never sit
|
||||
# behind a challenge) are exempt by construction: their vhosts never route
|
||||
# through Anubis. ComfyUI left this exempt set on 2026-08-31 and Grafana left
|
||||
# it on 2026-09-01 (OAuth2 relying-party mode against authentik): both now sit
|
||||
# behind the guard chain; Grafana authenticates users itself via the OAuth2
|
||||
# code flow, so no identity header is forwarded to it.
|
||||
# The pihole vhost keeps its basicauth edge; the hash placeholder below is
|
||||
# rendered by deploy.sh from CADDY_BASICAUTH_HASH (host .env, git-ignored) into
|
||||
# /home/dank/Containers/caddy/Caddyfile, which the container mounts.
|
||||
#
|
||||
# TLS: real Let's Encrypt certs. Site addresses carry no http:// prefix,
|
||||
# so Caddy serves HTTPS (and auto-redirects HTTP). Certs are issued via
|
||||
# dns-01 through the Porkbun module built into the franksdank/caddy-porkbun
|
||||
# image (caddy/Containerfile). That needs no inbound ports and works from the
|
||||
# LAN, where Pi-hole resolves these names to 192.168.0.143.
|
||||
|
||||
{
|
||||
# Admin stays ON: it binds to 127.0.0.1:2019 inside the container and the
|
||||
# port is not published, so nothing outside the container can reach it.
|
||||
# It powers the config file watcher, so a rendered-config change
|
||||
# hot-reloads without a container restart. 'admin off' once left a broken
|
||||
# basicauth hash sitting on disk unapplied for hours (2026-08-22).
|
||||
email admin@franksdank.cloud
|
||||
|
||||
# Solve ACME challenges for every site via Porkbun DNS (no port 80/443
|
||||
# reachability needed). Credentials come from the container environment
|
||||
# (compose.yaml, from the host .env — never committed).
|
||||
acme_dns porkbun {
|
||||
api_key {env.PORKBUN_API_KEY}
|
||||
api_secret_key {env.PORKBUN_API_SECRET_KEY}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Bare-IP access (health checks, convenience) ---
|
||||
:80 {
|
||||
redir https://www.franksdank.cloud/ 308
|
||||
}
|
||||
|
||||
# --- Homepage (Anubis → Authentik SSO) ---
|
||||
# www holds the homepage; the naked domain redirects onto it at the TLS edge.
|
||||
# A redirect needs no challenge, so unauthenticated naked traffic never enters
|
||||
# the auth chain. The proxy provider's external_host must match the host that
|
||||
# actually carries traffic (https://www.franksdank.cloud) or Authentik matches
|
||||
# zero providers and answers its error page instead.
|
||||
franksdank.cloud {
|
||||
redir https://www.franksdank.cloud/ 308
|
||||
}
|
||||
|
||||
www.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Legacy name: the old homepage name redirects straight onto www ---
|
||||
dankclaw.franksdank.cloud {
|
||||
redir https://www.franksdank.cloud/ 308
|
||||
}
|
||||
|
||||
# --- Legacy name: the media host has no Caddy of its own anymore ---
|
||||
dankserver.franksdank.cloud {
|
||||
redir https://www.franksdank.cloud/ 308
|
||||
}
|
||||
|
||||
# --- Authentik (SSO / OIDC provider) ---
|
||||
# Exempt by construction: the IdP login flow must be reachable without a
|
||||
# challenge, so this vhost never routes through Anubis.
|
||||
auth.franksdank.cloud {
|
||||
reverse_proxy authentik-server:9000
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# Services on this host (dankclaw, 192.168.0.143)
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
|
||||
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
|
||||
search.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
|
||||
ai.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.143:3005
|
||||
}
|
||||
|
||||
# --- Gitea (quadlet; published 3000) ---
|
||||
# Exempt by construction: Gitea has its own login, and git smart-HTTP
|
||||
# clients (git push/fetch over HTTPS) cannot solve JS challenges. The
|
||||
# policy would need a bypass for /info/refs + /git-*-pack paths if this
|
||||
# ever moves behind Anubis.
|
||||
gitea.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.143:3000
|
||||
}
|
||||
|
||||
# --- Grafana (guarded: Anubis → Authentik outpost gate; login inside Grafana) ---
|
||||
# Auth model = OAuth2 relying party against authentik (OAuth2Provider "grafana",
|
||||
# confidential client). The guard chain gates unauthenticated traffic; the
|
||||
# outpost carries no provider token in this deployment, so identity headers
|
||||
# never reach Grafana — RP mode needs none and works regardless of topology.
|
||||
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
|
||||
grafana.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Prometheus ---
|
||||
prometheus.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
|
||||
loki.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
|
||||
seerr.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
|
||||
# Content-cleaning proxy. Any authentik-authenticated user may use it (no
|
||||
# group gate). Egress rides the searxng gluetun tunnel; reached via the
|
||||
# host-published port 8082 (gluetun netns), not by container name.
|
||||
marreta.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Open Notebook (guarded: Anubis → Authentik; notebook-proxy provider, internal_host caddy:8017) ---
|
||||
# The podcast generator. ONE upstream carries the whole app: the image's own
|
||||
# Next.js layer rewrites /api/* to the FastAPI beside it (INTERNAL_API_URL,
|
||||
# default http://localhost:5055 inside the container), so a browser on this
|
||||
# vhost never needs the API port. That is why scripts/open-notebook.env sets
|
||||
# API_URL to exactly this origin — left unset, the app auto-detects
|
||||
# `<host-you-opened>:5055`, and 5055 on this host belongs to seerr — and why
|
||||
# the published API port sits on loopback. App-level login
|
||||
# (OPEN_NOTEBOOK_PASSWORD) stays on behind the chain: two owners, edge identity
|
||||
# and the app's own session.
|
||||
# Dialled by published LAN port, not container name: open-notebook is a
|
||||
# rootless quadlet and this Caddy lives in root's storage, so root's DNS does
|
||||
# not see the container (the same reason marreta and SearXNG are dialled by IP).
|
||||
notebook.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
||||
pihole.franksdank.cloud {
|
||||
basicauth {
|
||||
dankuser REPLACE_WITH_HASH
|
||||
}
|
||||
reverse_proxy 192.168.0.143:8083
|
||||
}
|
||||
|
||||
# --- Hermes WebUI (Hermex webui; host-netns container, published 8787) ------
|
||||
# Exempt by construction: the webui authenticates itself (HERMES_WEBUI_PASSWORD),
|
||||
# so no Anubis challenge and no Authentik proxy provider is required — nothing
|
||||
# here enters the http://:8016 auth chain. Internal-only by DNS absence: there is
|
||||
# no public Porkbun A record for this name, so only LAN clients (Pi-hole
|
||||
# addn-hosts → 192.168.0.143) can resolve it at all; DNS-01 issuance via the
|
||||
# Porkbun module needs no public record. Upstream is `host.containers.internal`
|
||||
# — netavark's host address, 169.254.1.2 — NOT the host LAN IP: hermes-webui is
|
||||
# `Network=host`, so 8787 is a host-netns bind and not a published port, and
|
||||
# netavark DNATs only published ports (192.168.0.143:9119, the dashboard below,
|
||||
# answers from this bridge; 192.168.0.143:8787 does not). Measured live
|
||||
# 2026-09-16 inside the Caddy container: 192.168.0.143:8787 and 10.89.1.1:8787
|
||||
# both REFUSED, host.containers.internal:8787 → 200.
|
||||
hermes.franksdank.cloud {
|
||||
reverse_proxy host.containers.internal:8787
|
||||
}
|
||||
|
||||
# --- Hermes dashboard (admin UI; published 9119 on dankclaw_default) --------
|
||||
# Exempt by construction: the app enforces its own complete auth — verified live
|
||||
# from Caddy's own network, an unauthenticated GET / returns 302 → /login?next=%2F
|
||||
# (uvicorn), and basic_auth.username + a scrypt password_hash are set in the
|
||||
# dashboard's /data/config.yaml. No outpost provider needed.
|
||||
hermesadmin.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.143:9119
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# Services on dankserver (192.168.0.235) — published LAN ports
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
|
||||
# --- Plex (existing, kept) ---
|
||||
# http:// on purpose: franksdank.casa is not hosted in the same Porkbun
|
||||
# account as franksdank.cloud, so the DNS-01 provider cannot issue a
|
||||
# Let's Encrypt cert for it. It stays plain-HTTP legacy; the canonical
|
||||
# name is plex.franksdank.cloud (HTTPS, below).
|
||||
http://plex.franksdank.casa {
|
||||
reverse_proxy 192.168.0.235:32400 {
|
||||
transport http {
|
||||
tls_insecure_skip_verify
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# PMS requires TLS from every non-loopback client: plain-HTTP off-loopback
|
||||
# is read and silently closed (diagnosed 2026-08-26), which surfaced as an
|
||||
# empty-body 502 here. Dial the upstream over TLS; PMS's self-signed cert is
|
||||
# skipped on this pinned-LAN hop only (client-facing LE TLS is untouched).
|
||||
# The web UI lives at /web/; / returns the XML MediaContainer index.
|
||||
plex.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.235:32400 {
|
||||
transport http {
|
||||
tls_insecure_skip_verify
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Calibre-Web ---
|
||||
books.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.235:8083
|
||||
}
|
||||
|
||||
# --- RomM ---
|
||||
romm.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Copyparty ---
|
||||
files.franksdank.cloud {
|
||||
reverse_proxy anubis:8923 {
|
||||
header_up X-Real-Ip {remote_host}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Jellyfin ---
|
||||
jelly.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.235:8096
|
||||
}
|
||||
|
||||
|
||||
# --- *arr Stack (exempt by construction: own auth) ---
|
||||
|
||||
prowlarr.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.112:9696
|
||||
}
|
||||
|
||||
radarr.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.112:7878
|
||||
}
|
||||
|
||||
# --- Sonarr: internal-only by construction (retired 2026-09-04) -----------
|
||||
# No vhost and no public DNS record for sonarr.franksdank.cloud: the TV-library
|
||||
# admin UI is reachable only on the LAN at 192.168.0.112:8989 (users request TV
|
||||
# through Seerr, which sits behind the guard chain; admins reach Sonarr's own
|
||||
# UI directly). A brief unguarded public vhost lived here and served the
|
||||
# zero-auth UI to the internet via manual DNS — retired.
|
||||
|
||||
lidarr.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.112:8686
|
||||
}
|
||||
|
||||
readarr.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.112:8787
|
||||
}
|
||||
|
||||
qbittorrent.franksdank.cloud {
|
||||
reverse_proxy 192.168.0.112:8080
|
||||
}
|
||||
|
||||
# --- Internal listener: auth decision layer ---------------------------------
|
||||
# Anubis dials http://caddy:8016 after verifying a client (TARGET in
|
||||
# dankclaw/anubis/quadlet/anubis.container). Host headers survive the hop.
|
||||
# Every request here is proxied whole-request to Authentik with Host
|
||||
# preserved. This build's embedded outpost routes ONLY on the ORIGINAL Host
|
||||
# against each proxy provider's external_host — meta headers are ignored and
|
||||
# cross-host absolute next= redirects are rejected by the flow's open-redirect
|
||||
# guard ("Request has been denied: Invalid next URL"), verified live.
|
||||
# Unauthenticated clients get the login flow served in place; authenticated
|
||||
# sessions are forwarded by the provider's internal_host back to
|
||||
# http://:8017 and routed straight to the app. Every host that may enter this
|
||||
# listener MUST have a proxy provider registered with the embedded outpost —
|
||||
# the Anubis guard set, the Caddy TLS edge, and the provider list stay in
|
||||
# lockstep. Exempt vhosts never enter this block.
|
||||
http://:8016 {
|
||||
reverse_proxy authentik-server:9000 {
|
||||
header_up X-Forwarded-Proto https
|
||||
}
|
||||
}
|
||||
|
||||
# --- Post-auth listener: where Authentik forwards authenticated traffic ----
|
||||
# The proxy provider's internal_host is http://caddy:8017, so every request
|
||||
# that passes authentication lands here (Host header preserved) and routes
|
||||
# straight to its app. This is the end of the chain — nothing dials back
|
||||
# into Authentik from here, so the chain cannot loop.
|
||||
http://:8017 {
|
||||
@home host www.franksdank.cloud
|
||||
handle @home {
|
||||
root * /srv/homepage
|
||||
# Deploys overwrite /srv/homepage in place; without revalidation a
|
||||
# browser keeps serving the pre-deploy CSS/JS/HTML. no-cache forces an
|
||||
# ETag round-trip on every load (a cheap 304 when unchanged).
|
||||
header Cache-Control "no-cache"
|
||||
# RBAC (2026-09-01): /admin requires membership in the homepage-admins
|
||||
# authentik group. In proxy-provider mode the outpost injects
|
||||
# X-Authentik-Groups on its upstream request to this listener and strips
|
||||
# any client-supplied X-Authentik-* headers first, so browsers cannot
|
||||
# spoof it. Regex is a substring match, insensitive to the group-list
|
||||
# separator.
|
||||
handle /admin* {
|
||||
@adminok header_regexp X-Authentik-Groups homepage-admins
|
||||
handle @adminok {
|
||||
try_files /admin.html
|
||||
file_server
|
||||
}
|
||||
respond "Forbidden: homepage admins only" 403
|
||||
}
|
||||
# User docs page (static). Anchors (#models etc.) are client-side,
|
||||
# so both /docs and /docs/ map onto the single docs.html.
|
||||
handle /docs {
|
||||
try_files /docs.html
|
||||
file_server
|
||||
}
|
||||
handle /docs/ {
|
||||
try_files /docs.html
|
||||
file_server
|
||||
}
|
||||
# /whoami echoes the outpost-injected identity headers so the page
|
||||
# can gate the admin button client-side (the /admin* 403 above stays
|
||||
# the real gate; this only decides what is shown). The outpost strips
|
||||
# client-supplied X-Authentik-* before injecting, so this is spoof-proof.
|
||||
handle /whoami {
|
||||
templates
|
||||
header Content-Type application/json
|
||||
respond "{\"user\":\"{header.X-Authentik-Username}\",\"groups\":\"{header.X-Authentik-Groups}\"}"
|
||||
}
|
||||
@data path /services.json /status.json /app.js /style.css /deployed.js /img/* /sfx/*
|
||||
handle @data {
|
||||
file_server
|
||||
}
|
||||
handle {
|
||||
try_files /index.html
|
||||
file_server
|
||||
}
|
||||
}
|
||||
|
||||
@search host search.franksdank.cloud
|
||||
handle @search {
|
||||
reverse_proxy 192.168.0.143:8081
|
||||
}
|
||||
|
||||
@loki host loki.franksdank.cloud
|
||||
handle @loki {
|
||||
reverse_proxy loki:3100
|
||||
}
|
||||
|
||||
@seerr host seerr.franksdank.cloud
|
||||
handle @seerr {
|
||||
reverse_proxy seerr:5055
|
||||
}
|
||||
|
||||
@grafana host grafana.franksdank.cloud
|
||||
handle @grafana {
|
||||
reverse_proxy grafana:3000
|
||||
}
|
||||
|
||||
@prometheus host prometheus.franksdank.cloud
|
||||
handle @prometheus {
|
||||
reverse_proxy prometheus:9090
|
||||
}
|
||||
|
||||
@romm host romm.franksdank.cloud
|
||||
handle @romm {
|
||||
reverse_proxy 192.168.0.235:8085
|
||||
}
|
||||
|
||||
@files host files.franksdank.cloud
|
||||
handle @files {
|
||||
reverse_proxy 192.168.0.235:3923
|
||||
}
|
||||
|
||||
@prowlarr host prowlarr.franksdank.cloud
|
||||
handle @prowlarr {
|
||||
reverse_proxy 192.168.0.112:9696
|
||||
}
|
||||
|
||||
@radarr host radarr.franksdank.cloud
|
||||
handle @radarr {
|
||||
reverse_proxy 192.168.0.112:7878
|
||||
}
|
||||
|
||||
@lidarr host lidarr.franksdank.cloud
|
||||
handle @lidarr {
|
||||
reverse_proxy 192.168.0.112:8686
|
||||
}
|
||||
|
||||
@readarr host readarr.franksdank.cloud
|
||||
handle @readarr {
|
||||
reverse_proxy 192.168.0.112:8787
|
||||
}
|
||||
|
||||
@qbittorrent host qbittorrent.franksdank.cloud
|
||||
handle @qbittorrent {
|
||||
reverse_proxy 192.168.0.112:8080
|
||||
}
|
||||
|
||||
@marreta host marreta.franksdank.cloud
|
||||
handle @marreta {
|
||||
reverse_proxy 192.168.0.143:8082
|
||||
}
|
||||
|
||||
@notebook host notebook.franksdank.cloud
|
||||
handle @notebook {
|
||||
# Bare proxy, like ai.franksdank.cloud: the app's SSE answers
|
||||
# (/api/search/ask, chat sessions) stream for minutes through this
|
||||
# same listener with no flush or timeout tuning, so nothing here
|
||||
# needs it — a read/write timeout on the route would be the thing
|
||||
# that cuts them.
|
||||
reverse_proxy 192.168.0.143:8502
|
||||
}
|
||||
}
|
||||
|
||||
# --- Optional: Jackett ---
|
||||
# jackett.franksdank.cloud {
|
||||
# reverse_proxy 192.168.0.235:9117
|
||||
# }
|
||||
|
||||
# --- Optional: Whisparr ---
|
||||
# whisparr.franksdank.cloud {
|
||||
# reverse_proxy 192.168.0.235:6969
|
||||
# }
|
||||
@@ -1,40 +0,0 @@
|
||||
# ComfyUI on the gfx1151 (AMD Strix Halo / Radeon 8060S) ROCm torch build.
|
||||
#
|
||||
# Base image: AMD's gfx1151-tagged vLLM image. It is the only verified build
|
||||
# where torch.cuda.is_available() is True on this APU (device capability
|
||||
# (11,5) = gfx1151). It already ships the matching ROCm torch family
|
||||
# (torch 2.9.1+rocm, torchvision 0.24.0+rocm, torchaudio 2.9.0+rocm), so we
|
||||
# must NOT let pip reinstall them: ComfyUI's requirements.txt lists an unpinned
|
||||
# `torch`, which would otherwise clobber the gfx1151 kernels with a default build.
|
||||
FROM docker.io/rocm/vllm:rocm7.12.0_gfx1151_ubuntu24.04_py3.12_pytorch_2.9.1_vllm_0.16.0
|
||||
|
||||
# Bake the ComfyUI code into the image (rebuild the image to update the code).
|
||||
RUN git clone --depth 1 https://github.com/comfyanonymous/ComfyUI /opt/ComfyUI
|
||||
WORKDIR /opt/ComfyUI
|
||||
|
||||
# Pre-create the user-data dirs. The host mounts its collections over them
|
||||
# (see the comfyui service in compose.yaml).
|
||||
RUN mkdir -p /opt/ComfyUI/models /opt/ComfyUI/input /opt/ComfyUI/output /opt/ComfyUI/custom_nodes
|
||||
|
||||
# Install ComfyUI's dependencies, but skip the torch family (already present
|
||||
# above as the gfx1151 ROCm builds). torchsde and the rest are kept.
|
||||
RUN grep -viE '^(torch|torchvision|torchaudio)([^a-z]|$)' requirements.txt > /tmp/req-notorch.txt \
|
||||
&& pip install --no-cache-dir -r /tmp/req-notorch.txt
|
||||
|
||||
# The city96/ComfyUI-GGUF custom node (loaded from the host's custom_nodes
|
||||
# volume) needs the gguf library to decode GGUF-quantized diffusion models.
|
||||
# sentencepiece and protobuf are its optional tokenizer deps. Baking them into
|
||||
# the image keeps the dependency across container recreations — the node itself
|
||||
# lives in the host volume, which persists, but the Python env does not.
|
||||
RUN pip install --no-cache-dir "gguf>=0.13.0" sentencepiece protobuf
|
||||
|
||||
# Bake the qwen_image21 patch script into the image. It is run by the
|
||||
# startup wrapper before ComfyUI starts, so the patch survives container
|
||||
COPY dankclaw/apply_qwen21_patch.py /opt/ComfyUI/apply_qwen21_patch.py
|
||||
COPY dankclaw/comfyui-start.sh /opt/ComfyUI/comfyui-start.sh
|
||||
RUN chmod +x /opt/ComfyUI/comfyui-start.sh
|
||||
|
||||
CMD ["/opt/ComfyUI/comfyui-start.sh"]
|
||||
EXPOSE 8188
|
||||
# v2: VAE 4D-to-5D latent expansion for video VAEs (qwen_image21)
|
||||
# v3: fix in_channels to use shape[0] (latent input dim) not shape[1] (hidden dim)
|
||||
@@ -1,18 +0,0 @@
|
||||
# Alloy + a static busybox.
|
||||
#
|
||||
# grafana/alloy has a shell but no wget/curl, so the compose healthcheck
|
||||
# cannot run anything inside the container. This image drops a static
|
||||
# busybox into it for the healthcheck command only. All Alloy behavior is
|
||||
# unchanged.
|
||||
#
|
||||
# Build on dankclaw:
|
||||
# podman build -t franksdank/alloy:latest -f dankclaw/alloy/Containerfile .
|
||||
|
||||
FROM docker.io/library/alpine:latest AS tools
|
||||
# busybox-static is statically linked: the target images have no dynamic
|
||||
# loader (loki is distroless), so the alpine default /bin/busybox (musl
|
||||
# dynamic) would fail with "no such file or directory".
|
||||
RUN apk add --no-cache busybox-static
|
||||
|
||||
FROM docker.io/grafana/alloy:latest
|
||||
COPY --from=tools /bin/busybox.static /usr/local/bin/busybox
|
||||
@@ -1,25 +0,0 @@
|
||||
{
|
||||
"bots": [
|
||||
{
|
||||
"name": "deny AI scrapers",
|
||||
"user_agent_regex": "(?i)^(GPTBot|ChatGPT-User|OAI-SearchBot|CCBot|anthropic-ai|ClaudeBot|Claude-Web|PerplexityBot|PerplexityUser|Google-Extended|Applebot-Extended|Bytespider|Amazonbot|meta-externalagent|DuckAssistBot|cohere-ai|YouBot|KimiBot)",
|
||||
"action": "DENY"
|
||||
},
|
||||
{
|
||||
"name": "serve anubis own endpoints",
|
||||
"path_regex": "^/\\.within\\.website",
|
||||
"action": "ALLOW"
|
||||
},
|
||||
{
|
||||
"name": "serve authentik outpost transport",
|
||||
"path_regex": "^/outpost\\.goauthentik\\.io/",
|
||||
"action": "ALLOW"
|
||||
},
|
||||
{
|
||||
"name": "challenge everything else",
|
||||
"path_regex": ".",
|
||||
"action": "CHALLENGE"
|
||||
}
|
||||
],
|
||||
"dnsbl": false
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
# Quadlet unit: Anubis bot-challenge proxy (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/anubis.container
|
||||
# Unit name: anubis.service (container name: anubis)
|
||||
#
|
||||
# Two-tier chain: the public Caddy listeners for the guarded vhosts proxy
|
||||
# everything to this container (port 8923, internal only — no published port,
|
||||
# so metrics on :9090 stay reachable only inside dankclaw_default); Anubis
|
||||
# verifies the client (challenge/DENY per policy) and forwards accepted
|
||||
# traffic back to Caddy's internal listener (TARGET below), where the normal
|
||||
# per-vhost routing — including Authentik forward_auth — resumes. Host headers
|
||||
# survive both hops, so vhost matching on the internal listener still works.
|
||||
#
|
||||
# Image pin: the image is pulled from ghcr.io/techarohq/anubis (source of
|
||||
# truth), but the unit runs it under a localhost/-aliased tag: on this host's
|
||||
# podman 5.7 quadlet path, registry-prefixed refs fail to resolve from local
|
||||
# storage at container create time ("image not known") even when present; the
|
||||
# localhost alias resolves (verified live 2026-08-28). v1.27.0 pinned 2026-09-02: --help
|
||||
# verified on the image itself (cookie-domain, ed25519-private-key-hex,
|
||||
# cookie-expiration-time default 168h, difficulty default 4); JSON policy
|
||||
# schema unchanged; X-Real-Ip is now REQUIRED (missing header => HTTP 500) —
|
||||
# the Caddy chain always sets it (bats seam enforces the header_up line).
|
||||
# Upgrading = pull new tag, re-tag the localhost alias, restart the unit.
|
||||
#
|
||||
# No secrets in this repo. COOKIE_DOMAIN makes the challenge cookie valid for
|
||||
# all *.franksdank.cloud vhosts: solve the proof-of-work once (e.g. on www)
|
||||
# and every guarded vhost honors it until expiry (default 168h, aligned with
|
||||
# authentik's 7-day session) or re-login. ED25519_PRIVATE_KEY_HEX pins the
|
||||
# JWT signing key (host file /home/dank/Containers/anubis/env, mode 0600,
|
||||
# gitea-style ${VAR} expansion via the [Service] EnvironmentFile): without
|
||||
# it every restart invalidates everyone's solved challenges. See
|
||||
# serverAppConfigs-docs/Infrastructure/Anubis.md.
|
||||
|
||||
[Unit]
|
||||
Description=Anubis bot-challenge proxy container
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=anubis
|
||||
Image=localhost/techarohq/anubis:v1.27.0
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
# flagenv maps env names to flag names (TARGET -> --target, POLICY_FNAME ->
|
||||
# --policy-fname). TARGET points back at Caddy's internal listener; the
|
||||
# container DNS name resolves on dankclaw_default.
|
||||
Environment=TARGET=http://caddy:8016
|
||||
Environment=POLICY_FNAME=/policy/policy.json
|
||||
Environment=COOKIE_DOMAIN=franksdank.cloud
|
||||
EnvironmentFile=/home/dank/Containers/anubis/env
|
||||
Environment=ED25519_PRIVATE_KEY_HEX=${ED25519_PRIVATE_KEY_HEX}
|
||||
Volume=/home/dank/serverAppConfigs/dankclaw/anubis/policy.json:/policy/policy.json:ro
|
||||
# The image ships no HEALTHCHECK (distroless ko-build: entrypoint is
|
||||
# /ko-app/anubis, no shell in the image), so the probe is declared here.
|
||||
# JSON exec form on purpose: this quadlet generator joins HealthCmd into a
|
||||
# single \x20-escaped argv element and podman then runs it as shell-form —
|
||||
# which needs a shell the distroless image does not have. A JSON array is
|
||||
# passed through verbatim and executes /ko-app/anubis directly.
|
||||
HealthCmd=["/ko-app/anubis","--healthcheck"]
|
||||
HealthInterval=15s
|
||||
HealthTimeout=5s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/anubis/env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,52 +0,0 @@
|
||||
import os
|
||||
|
||||
# Apply qwen_image21 detection patch
|
||||
f = "/opt/ComfyUI/comfy/model_detection.py"
|
||||
with open(f) as fh:
|
||||
code = fh.read()
|
||||
if "qwen_image21" not in code:
|
||||
qwen21_check = (
|
||||
" if '{}txt_in.text_norm.weight'.format(key_prefix) in state_dict_keys: # Qwen Image 2.1\n"
|
||||
" dit_config = {}\n"
|
||||
' dit_config["image_model"] = "qwen_image21"\n'
|
||||
" dit_config[\"in_channels\"] = state_dict['{}img_in.weight'.format(key_prefix)].shape[0]\n"
|
||||
" dit_config[\"num_layers\"] = count_blocks(state_dict_keys, '{}transformer_blocks.'.format(key_prefix) + '{}.')\n"
|
||||
" return dit_config\n"
|
||||
)
|
||||
anchor = " if '{}txt_norm.weight'.format(key_prefix) in state_dict_keys: # Qwen Image"
|
||||
code = code.replace(anchor, qwen21_check + "\n" + anchor)
|
||||
with open(f, "w") as fh:
|
||||
fh.write(code)
|
||||
print("Applied qwen_image21 detection patch")
|
||||
|
||||
# Add qwen_image21 to GGUF node arch list
|
||||
loader = "/opt/ComfyUI/custom_nodes/ComfyUI-GGUF/loader.py"
|
||||
if os.path.exists(loader):
|
||||
with open(loader) as fh:
|
||||
lcode = fh.read()
|
||||
if '"qwen_image21"' not in lcode:
|
||||
old = '"lumina2", "qwen_image"}'
|
||||
new = '"lumina2", "qwen_image", "qwen_image21"}'
|
||||
if old in lcode:
|
||||
lcode = lcode.replace(old, new)
|
||||
with open(loader, "w") as fh:
|
||||
fh.write(lcode)
|
||||
print("Applied qwen_image21 arch list patch")
|
||||
|
||||
# Patch VAE decode to expand 4D latents to 5D for video VAEs (latent_dim=3)
|
||||
sd = "/opt/ComfyUI/comfy/sd.py"
|
||||
with open(sd) as fh:
|
||||
sdc = fh.read()
|
||||
if "self.latent_dim == 3 and samples_in.ndim == 4" not in sdc:
|
||||
anchor = " if self.latent_dim == 2 and samples_in.ndim == 5:\n samples_in = samples_in[:, :, 0]"
|
||||
replacement = (
|
||||
" if self.latent_dim == 2 and samples_in.ndim == 5:\n"
|
||||
" samples_in = samples_in[:, :, 0]\n"
|
||||
" elif self.latent_dim == 3 and samples_in.ndim == 4:\n"
|
||||
" samples_in = samples_in.unsqueeze(2)"
|
||||
)
|
||||
if anchor in sdc:
|
||||
sdc = sdc.replace(anchor, replacement)
|
||||
with open(sd, "w") as fh:
|
||||
fh.write(sdc)
|
||||
print("Applied VAE 4D-to-5D latent expansion patch")
|
||||
@@ -1,40 +0,0 @@
|
||||
# Quadlet unit: Authentik LDAP outpost (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/authentik-outpost-ldap.container
|
||||
# Unit name: authentik-outpost-ldap.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# Serves the authentik LDAP provider ("Jellyfin LDAP", Base DN
|
||||
# dc=franksdank,dc=cloud) so dankserver's Jellyfin can authenticate its users
|
||||
# via the LDAP-Auth plugin. The type-specific ghcr.io/goauthentik/ldap image
|
||||
# listens on 3389 (ldap) / 6636 (ldaps) inside the container; published to
|
||||
# host 389/636 for cross-host binds from dankserver. The outpost token lives
|
||||
# in the host .env (AUTHENTIK_LDAP_OUTPOST_TOKEN), never in this repo. The
|
||||
# outpost dials core over websocket at AUTHENTIK_HOST; auth. is guard-chain
|
||||
# exempt (IdP).
|
||||
|
||||
[Unit]
|
||||
Description=Authentik LDAP outpost container
|
||||
After=network.target authentik-server.service
|
||||
Wants=authentik-server.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-outpost-ldap
|
||||
Network=dankclaw_default
|
||||
# Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
|
||||
# the tag's nested manifest list, so pin the single-platform digest).
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
|
||||
Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
|
||||
Environment=AUTHENTIK_INSECURE=false
|
||||
Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}
|
||||
PublishPort=389:3389
|
||||
PublishPort=636:6636
|
||||
User=0:0
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,44 +0,0 @@
|
||||
# Quadlet unit: Authentik outpost (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/authentik-outpost.container
|
||||
# Unit name: authentik-outpost.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# In authentik 2026.8.x the proxy-provider URL space (/outpost.goauthentik.io/*)
|
||||
# is served by the Rust worker binary, not the server — verified live: with a
|
||||
# tenant-matching Host header, /if/flow/... answers 200 on the server while
|
||||
# /auth/caddy still 404s. This container runs the same image as the server and
|
||||
# worker units; Exec=worker selects the Rust worker role, and the
|
||||
# AUTHENTIK_HOST/AUTHENTIK_TOKEN env flips it into outpost mode: it dials back
|
||||
# to core over websocket (ws/outpost/<uuid>/) and serves forward-auth for Caddy
|
||||
# on its own 9000. No published port — Caddy reaches it by container DNS on
|
||||
# dankclaw_default; the token lives in the host .env, never in this repo.
|
||||
|
||||
[Unit]
|
||||
Description=Authentik outpost container
|
||||
After=network.target authentik-postgres.service authentik-server.service
|
||||
Wants=authentik-server.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-outpost
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3
|
||||
Exec=worker
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
|
||||
Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__USER=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
|
||||
Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
|
||||
Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
|
||||
Environment=AUTHENTIK_INSECURE=false
|
||||
Environment=AUTHENTIK_TOKEN=${AUTHENTIK_OUTPOST_TOKEN}
|
||||
Volume=/home/dank/Containers/authentik/data:/data
|
||||
User=0:0
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,42 +0,0 @@
|
||||
# Quadlet unit: Authentik PostgreSQL 16 (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/authentik-postgres.container
|
||||
# Unit name: authentik-postgres.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# Replaces the compose service retired when this stack went boot-persistent
|
||||
# (rootless Podman does not auto-start bare containers at boot). Secrets stay
|
||||
# out of this file: systemd expands ${AUTHENTIK_PG_PASS} from the git-ignored
|
||||
# host .env at start time (same pattern as gitea-postgres).
|
||||
|
||||
[Unit]
|
||||
Description=Authentik PostgreSQL container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-postgres
|
||||
Image=docker.io/library/postgres:16-alpine
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=POSTGRES_DB=authentik
|
||||
Environment=POSTGRES_USER=authentik
|
||||
Environment=POSTGRES_PASSWORD=${AUTHENTIK_PG_PASS}
|
||||
Volume=/home/dank/Containers/authentik/postgres:/var/lib/postgresql/data
|
||||
# Unquoted argv: this quadlet generator space-splits HealthCmd into the
|
||||
# healthcheck's argv array, so a quoted probe would arrive mangled (the old
|
||||
# `/bin/sh -c "pg_isready..."` form arrived as /bin/sh + broken args and
|
||||
# failed with exit 1 forever — the health status was a false negative while
|
||||
# the DB itself accepted connections). Bare argv works under both exec forms.
|
||||
HealthCmd=pg_isready -d authentik -U authentik
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=5
|
||||
HealthStartPeriod=20s
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,45 +0,0 @@
|
||||
# Quadlet unit: Authentik core + embedded Caddy outpost (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/authentik-server.container
|
||||
# Unit name: authentik-server.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The embedded outpost serves /outpost.goauthentik.io/* including the
|
||||
# /auth/caddy forward_auth endpoint the Caddyfile chains to — no outpost token
|
||||
# exists for this deployment by design. Resolved by name (A2A DNS) from every
|
||||
# container on dankclaw_default, including caddy. The 9000 publish is
|
||||
# loopback-only on purpose: LAN clients reach Authentik through Caddy, never
|
||||
# around it; the loopback publish exists so host-side tooling can probe
|
||||
# /-/health/. Runs as the image default user (same userns mapping the compose
|
||||
# era used, so the existing /data ownership carries over unchanged).
|
||||
|
||||
[Unit]
|
||||
Description=Authentik server container
|
||||
After=network.target authentik-postgres.service
|
||||
Wants=authentik-postgres.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-server
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3
|
||||
Exec=server
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
|
||||
Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__USER=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
|
||||
Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
|
||||
PublishPort=127.0.0.1:9000:9000
|
||||
# Probe must contain no quotes: this host's quadlet v1 generator space-splits
|
||||
# HealthCmd into the healthcheck argv array, and quoted probes arrive mangled
|
||||
# (verified live 2026-08-28). `python /manage.py check --database default`
|
||||
# exits nonzero iff Django (incl. the DB connection) cannot serve.
|
||||
HealthCmd=python /manage.py check --database default
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,37 +0,0 @@
|
||||
# Quadlet unit: Authentik worker — background tasks, email, webhook/log outbox
|
||||
# (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/authentik-worker.container
|
||||
# Unit name: authentik-worker.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# Same image as the server; the Exec= command selects the worker role. Runs as
|
||||
# user 0:0 (rootless: maps to the invoking host user, matching the compose-era
|
||||
# semantics). No published ports — the worker talks to PostgreSQL in-network
|
||||
# and has no listener of its own.
|
||||
|
||||
[Unit]
|
||||
Description=Authentik worker container
|
||||
After=network.target authentik-postgres.service authentik-server.service
|
||||
Wants=authentik-server.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-worker
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3
|
||||
Exec=worker
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=AUTHENTIK_POSTGRESQL__HOST=authentik-postgres
|
||||
Environment=AUTHENTIK_POSTGRESQL__NAME=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__USER=authentik
|
||||
Environment=AUTHENTIK_POSTGRESQL__PASSWORD=${AUTHENTIK_PG_PASS}
|
||||
Environment=AUTHENTIK_SECRET_KEY=${AUTHENTIK_SECRET_KEY}
|
||||
Volume=/home/dank/Containers/authentik/data:/data
|
||||
User=0:0
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,22 +0,0 @@
|
||||
# Caddy with the Porkbun DNS provider (github.com/caddy-dns/porkbun).
|
||||
#
|
||||
# The porkbun module lets Caddy solve Let's Encrypt DNS-01 challenges
|
||||
# directly through the Porkbun API, so the wildcard cert for
|
||||
# *.franksdank.cloud / *.franksdank.casa can be issued and renewed without
|
||||
# any port 80/443 reachability from the internet (see the Caddyfile in
|
||||
# ../Caddyfile and serverAppConfigs-docs/Infrastructure/LAN-DNS.md).
|
||||
#
|
||||
# Build on dankclaw (podman rootless, same as every other container here):
|
||||
# podman build -t franksdank/caddy-porkbun:latest -f dankclaw/caddy/Containerfile .
|
||||
#
|
||||
# The first build runs an xcaddy Go compilation and takes a few minutes.
|
||||
# Rebuild only when Caddy or the porkbun module must be updated.
|
||||
|
||||
FROM docker.io/library/caddy:2-builder AS builder
|
||||
|
||||
RUN xcaddy build \
|
||||
--with github.com/caddy-dns/porkbun \
|
||||
--output /usr/bin/caddy-porkbun
|
||||
|
||||
FROM docker.io/library/caddy:2
|
||||
COPY --from=builder /usr/bin/caddy-porkbun /usr/bin/caddy
|
||||
@@ -1,7 +0,0 @@
|
||||
# Caddy environment (placeholders only). The deploy writes the live file at
|
||||
# /home/dank/Containers/caddy/env from the checkout's .env; the quadlet reads
|
||||
# it via EnvironmentFile. Consumed by the Caddyfile's acme_dns block
|
||||
# (DNS-01 wildcard cert issuance). PORKBUN_API_SECRET_KEY is the name the
|
||||
# caddy-dns/porkbun module expects.
|
||||
PORKBUN_API_KEY=changeme
|
||||
PORKBUN_API_SECRET_KEY=changeme
|
||||
@@ -1,54 +0,0 @@
|
||||
# Quadlet unit: canonical Caddy reverse proxy + ACME (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/caddy.container
|
||||
# Unit name: caddy.service (container name: caddy)
|
||||
#
|
||||
# The homelab's single Caddy: it fronts every service on both hosts and issues
|
||||
# Let's Encrypt wildcard certs via DNS-01 (Porkbun). It left compose.yaml on
|
||||
# 2026-08-26 after a reboot exposed that the bare compose container never came
|
||||
# back: rootless Podman has no boot-time auto-start for bare containers, and
|
||||
# `restart: unless-stopped` does not survive the explicit stop that shutdown
|
||||
# performs. See AGENTS.md "Boot Persistence & Self-Healing" and
|
||||
# serverAppConfigs-docs/Infrastructure/Caddy.md.
|
||||
#
|
||||
# It joins the existing compose network (dankclaw_default) so the Caddyfile's
|
||||
# container-name upstreams (authentik-server, grafana, ...) still resolve; if
|
||||
# no compose service has created that network yet at boot, quadlet creates it
|
||||
# first and compose reuses it by name later. The image is built locally from
|
||||
# dankclaw/caddy/Containerfile and exists only on this host, so Pull=never: a
|
||||
# boot must not stall on a registry that does not have it. Porkbun credentials
|
||||
# come from /home/dank/Containers/caddy/env (deploy writes it from the
|
||||
# checkout's .env; secrets never enter the repo).
|
||||
|
||||
[Unit]
|
||||
Description=Caddy reverse proxy + ACME (Porkbun DNS-01) container
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=caddy
|
||||
Image=franksdank/caddy-porkbun:latest
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
PublishPort=0.0.0.0:80:80/tcp
|
||||
PublishPort=0.0.0.0:443:443/tcp
|
||||
EnvironmentFile=/home/dank/Containers/caddy/env
|
||||
Volume=/home/dank/Containers/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
|
||||
Volume=/home/dank/Containers/caddy/data:/data
|
||||
Volume=/home/dank/Containers/caddy/config:/config
|
||||
# The homepage: the host's /srv/homepage (deploy-managed, not the repo's
|
||||
# shared/homepage).
|
||||
Volume=/srv/homepage:/srv/homepage:ro
|
||||
# The bare-IP catch-all answers 302 (curl -f passes on <400); this proves
|
||||
# Caddy itself is serving without touching any upstream.
|
||||
HealthCmd=curl -sf -o /dev/null http://localhost:80/
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Apply qwen_image21 patches before starting ComfyUI
|
||||
python3 /opt/ComfyUI/apply_qwen21_patch.py 2>/dev/null
|
||||
exec python main.py --listen 0.0.0.0 --port 8188
|
||||
@@ -1,21 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""ComfyUI container healthcheck: the UI must answer on :8188.
|
||||
|
||||
Runs INSIDE the container, where the UI listens (the unit publishes that port on
|
||||
the host loopback for the homepage status card). ComfyUI only answers once
|
||||
torch, the custom nodes and the graph are loaded, so a 200 here means the
|
||||
service is usable rather than merely a live process — the distinction that
|
||||
matters, because the failure this replaced was a container that stayed "up"
|
||||
while dying on every start.
|
||||
|
||||
HealthCmd in the quadlet splits on spaces into an argv array (podman 5.x v1
|
||||
generator), which is why this is a file and not an inline `python3 -c`.
|
||||
"""
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen("http://127.0.0.1:8188/", timeout=10) as resp:
|
||||
sys.exit(0 if resp.status == 200 else 1)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
@@ -1,70 +0,0 @@
|
||||
# Quadlet unit: ComfyUI (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/comfyui.container
|
||||
# Unit name: comfyui.service (container name: comfyui)
|
||||
#
|
||||
# WHY THIS FILE EXISTS. ComfyUI ran from `dankclaw/compose.yaml` through a
|
||||
# hand-written `~/.config/systemd/user/comfyui.service` that called
|
||||
# `podman compose up -d`. Its container was created 2026-09-15 14:12, BEFORE the
|
||||
# GPU declarations were added to the compose service, and `restart:
|
||||
# unless-stopped` then kept restarting that stale container — so it started
|
||||
# without /dev/kfd, without /dev/dri and without keep-groups, died on
|
||||
# `RuntimeError: No HIP GPUs are available`, and had restarted 9,146 times when
|
||||
# this was found. A recreate via compose would have fixed the container; a unit
|
||||
# fixes the class: one owner per container name, repo-tracked, boot-persistent
|
||||
# through the generator.
|
||||
#
|
||||
# GPU ACCESS. /dev/kfd + /dev/dri plus keep-groups. A literal `render` group
|
||||
# entry cannot work under rootless Podman (the group name does not resolve
|
||||
# inside the user namespace), and the companion udev rule
|
||||
# (dankclaw/udev/99-comfyui-gpu-access.rules) keeps those nodes MODE 0666 so a
|
||||
# user-unit spawn — which carries no supplementary groups — can still open them.
|
||||
# Both halves are asserted by tests/comfyui_rocm.bats.
|
||||
#
|
||||
# EXPOSURE. LAN-reachable, no auth. ComfyUI has no authentication and executes
|
||||
# arbitrary node graphs. It is internal-only (no public vhost, no guard chain);
|
||||
# it is reached on the LAN via the Pi-hole DNS record
|
||||
# (comfyui.franksdank.cloud -> 192.168.0.143, in dankclaw/pihole/addn-hosts),
|
||||
# which resolves to this host's IP — so the bind is widened from loopback to
|
||||
# 0.0.0.0:8188. The homepage admin card links to that URL and probes
|
||||
# 127.0.0.1:8188.
|
||||
#
|
||||
# DATA. Models, input, output and custom_nodes live under
|
||||
# /home/dank/comfyui-data (host state, not in the repo).
|
||||
[Unit]
|
||||
Description=ComfyUI (gfx1151 ROCm image generation)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=comfyui
|
||||
Image=localhost/comfyui-rocm:gfx1151
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
PublishPort=0.0.0.0:8188:8188
|
||||
AddDevice=/dev/kfd
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
Volume=/home/dank/comfyui-data/models:/opt/ComfyUI/models
|
||||
Volume=/home/dank/comfyui-data/input:/opt/ComfyUI/input
|
||||
Volume=/home/dank/comfyui-data/output:/opt/ComfyUI/output
|
||||
Volume=/home/dank/comfyui-data/custom_nodes:/opt/ComfyUI/custom_nodes
|
||||
Volume=/home/dank/serverAppConfigs/dankclaw/comfyui/comfyui-healthcheck.py:/usr/local/bin/comfyui-healthcheck.py:ro
|
||||
Exec=/opt/ComfyUI/comfyui-start.sh
|
||||
# The probe lives in a file because the podman 5.x v1 generator splits
|
||||
# HealthCmd on spaces into an argv array; an inline `python3 -c "..."` becomes
|
||||
# a puzzle of quoting that fails silently.
|
||||
HealthCmd=python3 /usr/local/bin/comfyui-healthcheck.py
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
# First start loads torch and the node graph; give it room before the probe
|
||||
# counts against it.
|
||||
HealthStartPeriod=300s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,64 +0,0 @@
|
||||
{
|
||||
"3": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Positive prompt" },
|
||||
"inputs": {
|
||||
"text": "A detailed illustration of a 1930s Soviet industrial design bureau at night: a well-measured engineer at a drafting table, a warm desk lamp, rolled blueprints, gears and calipers on the desk, deep red and cream palette, gouache painterly style, intricate mechanical detail, soft shadows",
|
||||
"clip": [ "6", 0 ]
|
||||
}
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Negative prompt" },
|
||||
"inputs": {
|
||||
"text": "lowres, blurry, deformed, extra limbs, watermark, signature, jpeg artifacts",
|
||||
"clip": [ "6", 0 ]
|
||||
}
|
||||
},
|
||||
"5": {
|
||||
"class_type": "UNETLoader",
|
||||
"_meta": { "title": "Qwen Image 2.1 (int8)" },
|
||||
"inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
|
||||
},
|
||||
"6": {
|
||||
"class_type": "CLIPLoader",
|
||||
"_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
|
||||
"inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
|
||||
},
|
||||
"7": {
|
||||
"class_type": "VAELoader",
|
||||
"_meta": { "title": "VAE" },
|
||||
"inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
|
||||
},
|
||||
"8": {
|
||||
"class_type": "EmptySD3LatentImage",
|
||||
"_meta": { "title": "Empty Latent 1024x1024" },
|
||||
"inputs": { "width": 1024, "height": 1024, "batch_size": 1 }
|
||||
},
|
||||
"9": {
|
||||
"class_type": "KSampler",
|
||||
"_meta": { "title": "KSampler" },
|
||||
"inputs": {
|
||||
"model": [ "5", 0 ],
|
||||
"seed": 1930,
|
||||
"steps": 20,
|
||||
"cfg": 1.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "simple",
|
||||
"positive": [ "3", 0 ],
|
||||
"negative": [ "4", 0 ],
|
||||
"latent_image": [ "8", 0 ],
|
||||
"denoise": 1.0
|
||||
}
|
||||
},
|
||||
"10": {
|
||||
"class_type": "VAEDecode",
|
||||
"_meta": { "title": "VAE Decode" },
|
||||
"inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
|
||||
},
|
||||
"11": {
|
||||
"class_type": "SaveImage",
|
||||
"_meta": { "title": "Save Image (art)" },
|
||||
"inputs": { "filename_prefix": "qwen21-art", "images": [ "10", 0 ] }
|
||||
}
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
{
|
||||
"3": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Positive prompt (legible text)" },
|
||||
"inputs": {
|
||||
"text": "Clean minimalist movie-poster design, dark charcoal background with a subtle vintage grain. Centered large bold white sans-serif text reading \"SOLIDARITY OF THE SOVIET COMPUTER SYSTEMS\", perfectly legible, crisp letterforms, even letter spacing. Below it, a smaller line of yellow text reading \"DANKCLAW DANKSERVER EST 2026\", equally legible. Flat graphic design, high contrast, generous margins, no other words, no watermark",
|
||||
"clip": [ "6", 0 ]
|
||||
}
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"_meta": { "title": "Negative prompt" },
|
||||
"inputs": {
|
||||
"text": "blurry, illegible, misspelled, warped letters, extra text, watermark, signature, lowres, jpeg artifacts",
|
||||
"clip": [ "6", 0 ]
|
||||
}
|
||||
},
|
||||
"5": {
|
||||
"class_type": "UNETLoader",
|
||||
"_meta": { "title": "Qwen Image 2.1 (int8)" },
|
||||
"inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
|
||||
},
|
||||
"6": {
|
||||
"class_type": "CLIPLoader",
|
||||
"_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
|
||||
"inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
|
||||
},
|
||||
"7": {
|
||||
"class_type": "VAELoader",
|
||||
"_meta": { "title": "VAE" },
|
||||
"inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
|
||||
},
|
||||
"8": {
|
||||
"class_type": "EmptySD3LatentImage",
|
||||
"_meta": { "title": "Empty Latent 832x1216 poster" },
|
||||
"inputs": { "width": 832, "height": 1216, "batch_size": 1 }
|
||||
},
|
||||
"9": {
|
||||
"class_type": "KSampler",
|
||||
"_meta": { "title": "KSampler" },
|
||||
"inputs": {
|
||||
"model": [ "5", 0 ],
|
||||
"seed": 2026,
|
||||
"steps": 25,
|
||||
"cfg": 1.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "simple",
|
||||
"positive": [ "3", 0 ],
|
||||
"negative": [ "4", 0 ],
|
||||
"latent_image": [ "8", 0 ],
|
||||
"denoise": 1.0
|
||||
}
|
||||
},
|
||||
"10": {
|
||||
"class_type": "VAEDecode",
|
||||
"_meta": { "title": "VAE Decode" },
|
||||
"inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
|
||||
},
|
||||
"11": {
|
||||
"class_type": "SaveImage",
|
||||
"_meta": { "title": "Save Image (text poster)" },
|
||||
"inputs": { "filename_prefix": "qwen21-text", "images": [ "10", 0 ] }
|
||||
}
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
# Gitea stack secrets. Copy to /opt/gitea/.env on dankclaw (mode 0600).
|
||||
# The live password never enters this repo.
|
||||
|
||||
GITEA_DB_PASSWORD=changeme
|
||||
@@ -1,31 +0,0 @@
|
||||
# Quadlet unit: Gitea PostgreSQL (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/gitea-postgres.container
|
||||
# Unit name: gitea-postgres.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The DB password lives in /home/dank/Containers/gitea/env on the host (mode 0600).
|
||||
# This file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gitea.md.
|
||||
# The [Service] EnvironmentFile= line lets systemd expand
|
||||
# ${GITEA_DB_PASSWORD} in the generated ExecStart.
|
||||
|
||||
[Unit]
|
||||
Description=Gitea PostgreSQL container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=gitea-postgres
|
||||
Network=gitea.network
|
||||
Image=docker.io/library/postgres:16-alpine
|
||||
EnvironmentFile=/home/dank/Containers/gitea/env
|
||||
Environment=POSTGRES_USER=gitea
|
||||
Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD}
|
||||
Environment=POSTGRES_DB=gitea
|
||||
Volume=/home/dank/Containers/gitea/postgres:/var/lib/postgresql/data
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/gitea/env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,44 +0,0 @@
|
||||
# Quadlet unit: Gitea web server (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/gitea-server.container
|
||||
# Unit name: gitea-server.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The [Service] EnvironmentFile= line lets systemd expand
|
||||
# ${GITEA_DB_PASSWORD} in the generated ExecStart.
|
||||
# The DB password lives in /home/dank/Containers/gitea/env on the host (mode 0600).
|
||||
# This file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gitea.md.
|
||||
#
|
||||
# Ports (live): 3000 = HTTP UI/API, 2222 = git SSH. The docs table in
|
||||
# Dankclaw Stack.md historically said 3001; the live host port is 3000.
|
||||
|
||||
[Unit]
|
||||
Description=Gitea web server container
|
||||
Wants=gitea-postgres.service
|
||||
After=gitea-postgres.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=gitea-server
|
||||
Network=gitea.network
|
||||
Image=docker.io/gitea/gitea:latest
|
||||
EnvironmentFile=/home/dank/Containers/gitea/env
|
||||
Environment=DATABASE_TYPE=postgres
|
||||
Environment=DATABASE_HOST=gitea-postgres
|
||||
Environment=DATABASE_NAME=gitea
|
||||
Environment=DATABASE_USER=gitea
|
||||
Environment=DATABASE_PASSWD=${GITEA_DB_PASSWORD}
|
||||
Environment=DATABASE_PORT=5432
|
||||
Environment=USER_UID=1000
|
||||
Environment=USER_GID=1000
|
||||
PublishPort=0.0.0.0:3000:3000
|
||||
PublishPort=2222:22
|
||||
Volume=/home/dank/Containers/gitea/data:/data
|
||||
Volume=/etc/timezone:/etc/timezone:ro
|
||||
Volume=/etc/localtime:/etc/localtime:ro
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/gitea/env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,10 +0,0 @@
|
||||
# Quadlet network: private bridge for the Gitea stack (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/gitea.network
|
||||
# Unit name: gitea-network.service (podman network name: systemd-gitea)
|
||||
#
|
||||
# DNS is enabled by default for quadlet networks. This matters: the
|
||||
# automatic "podman" default network has dns_enabled=false, so the
|
||||
# gitea-postgres and gitea-server container names would not resolve.
|
||||
|
||||
[Network]
|
||||
@@ -1,100 +0,0 @@
|
||||
# Hermes configuration backup / non-secret baseline.
|
||||
#
|
||||
# Source on Dankclaw: /home/dank/Containers/hermes/data/config.yaml
|
||||
# This file is a sanitized Git backup, not a complete replacement for the
|
||||
# live Hermes state. Do not add dashboard password hashes, phone numbers,
|
||||
# OAuth credentials, API keys, sessions, memories, or generated state here.
|
||||
# The live directory is bind-mounted into the containers as /data.
|
||||
|
||||
model:
|
||||
provider: dankclaw-large
|
||||
base_url: http://192.168.0.143:7770/v1
|
||||
api_key: not-needed
|
||||
default: frank-dragoon
|
||||
|
||||
providers:
|
||||
dankclaw-large:
|
||||
api: http://192.168.0.143:7770/v1
|
||||
discover_models: false
|
||||
models:
|
||||
frank-dragoon:
|
||||
context_length: 262144
|
||||
dankserver-small:
|
||||
api: http://192.168.0.235:7770/v1
|
||||
discover_models: false
|
||||
models:
|
||||
frank-medium:
|
||||
context_length: 114688
|
||||
|
||||
# Auxiliary routing from the live configuration. Named providers route via
|
||||
# their `api:` URL; `custom` entries route by their explicit `base_url`.
|
||||
auxiliary:
|
||||
vision: {provider: dankclaw-large, model: frank-dragoon}
|
||||
web_extract: {provider: dankserver-small, model: frank-medium}
|
||||
compression: {provider: dankserver-small, model: frank-medium}
|
||||
skills_hub: {provider: dankserver-small, model: frank-medium}
|
||||
approval: {provider: dankserver-small, model: frank-medium}
|
||||
mcp: {provider: dankserver-small, model: frank-medium}
|
||||
# Title generation goes out over the Responses transport on purpose: Hermes
|
||||
# hardcodes response_format=json_schema on this one call, no local back-end
|
||||
# implements constrained decoding (it is a 400), and this transport is the
|
||||
# only one that drops the field. reasoning.effort=none is required with it —
|
||||
# the call caps output at 64 tokens, which a thinking pass would consume
|
||||
# entirely, leaving an empty title.
|
||||
title_generation:
|
||||
provider: dankserver-small
|
||||
model: frank-medium
|
||||
api_mode: codex_responses
|
||||
extra_body:
|
||||
reasoning:
|
||||
effort: none
|
||||
memory_query_rewrite: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
|
||||
tts_audio_tags: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
|
||||
triage_specifier: {provider: dankclaw-large, model: frank-dragoon}
|
||||
kanban_decomposer: {provider: dankclaw-large, model: frank-dragoon}
|
||||
profile_describer: {provider: dankserver-small, model: frank-medium}
|
||||
curator: {provider: dankserver-small, model: frank-medium}
|
||||
|
||||
dashboard:
|
||||
# basic_auth intentionally omitted; its password_hash remains live-only.
|
||||
font: atkinson-hyperlegible
|
||||
moa:
|
||||
presets:
|
||||
default:
|
||||
reference_models:
|
||||
- provider: openrouter
|
||||
model: deepseek/deepseek-v4-pro
|
||||
enabled: false
|
||||
degraded_reference_policy: loud
|
||||
fanout: user_turn
|
||||
reference_models:
|
||||
- provider: openrouter
|
||||
model: deepseek/deepseek-v4-pro
|
||||
enabled: false
|
||||
aggregator:
|
||||
provider: openrouter
|
||||
model: anthropic/claude-opus-4.8
|
||||
degraded_reference_policy: loud
|
||||
max_tokens: 4096
|
||||
fanout: user_turn
|
||||
enabled: true
|
||||
skills:
|
||||
disabled: []
|
||||
onboarding:
|
||||
seen:
|
||||
busy_input_prompt: true
|
||||
_config_version: 34
|
||||
platforms:
|
||||
signal:
|
||||
# Live config has the Signal adapter disabled; http_url points at the
|
||||
# host signal-cli daemon. The account phone number stays live-only.
|
||||
enabled: false
|
||||
extra:
|
||||
http_url: http://192.168.0.143:8080
|
||||
mcp_servers:
|
||||
chrome-devtools:
|
||||
command: /usr/local/bin/npx
|
||||
args:
|
||||
- -y
|
||||
- chrome-devtools-mcp@latest
|
||||
- --browserUrl=http://127.0.0.1:9222
|
||||
@@ -1,38 +0,0 @@
|
||||
# Quadlet unit: Chrome DevTools Protocol daemon for the hermes agent (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/chrome-devtools-protocol.container
|
||||
# Unit name: chrome-devtools-protocol.service (container name: chrome-devtools-protocol)
|
||||
#
|
||||
# Headless Chrome exposing raw CDP over http://127.0.0.1:9222. The hermes
|
||||
# agent's chrome-devtools MCP client (mcp_servers.chrome-devtools in
|
||||
# /data/config.yaml) spawns npx chrome-devtools-mcp with --browserUrl pointing
|
||||
# at this endpoint and drives the browser over it. Both consumers — the Signal
|
||||
# gateway and the webui server — run in the host network namespace, so a
|
||||
# loopback-only bind is reachable by exactly the processes that may drive the
|
||||
# browser and by nobody else on the LAN or WAN.
|
||||
#
|
||||
# Entrypoint override drops the image's socat front (which would bind
|
||||
# 0.0.0.0:9222 in the host namespace and hand browser control to the whole
|
||||
# LAN). --no-sandbox is required because rootless podman grants Chrome no user
|
||||
# namespace for its sandbox; the content processes therefore run as the uid
|
||||
# that already owns this container, capping the blast radius of a hostile page.
|
||||
# Image pinned by digest: upgrades are a deliberate unit edit + redeploy, not a
|
||||
# tag drift at pull time.
|
||||
|
||||
[Unit]
|
||||
Description=Chrome DevTools Protocol daemon (headless-shell) for hermes
|
||||
|
||||
[Container]
|
||||
ContainerName=chrome-devtools-protocol
|
||||
Network=host
|
||||
Image=docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04
|
||||
Entrypoint=/headless-shell/headless-shell
|
||||
Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222
|
||||
ShmSize=1073741824
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,33 +0,0 @@
|
||||
# Quadlet unit: Hermes web dashboard (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/hermes-dashboard.container
|
||||
# Unit name: hermes-dashboard.service (container name: hermes-dashboard)
|
||||
#
|
||||
# Management UI for the agent (config, API keys, sessions). Published on
|
||||
# 9119 on network dankclaw_default; Caddy's hermesadmin.franksdank.cloud vhost
|
||||
# reverse-proxies this port directly (no guard chain) because the dashboard
|
||||
# enforces its own basic-auth login from /data/config.yaml
|
||||
# (basic_auth.username + scrypt password_hash), making it exempt-by-construction.
|
||||
# The name has no public DNS record, so the vhost resolves on the LAN only.
|
||||
# Shares /data with the gateway: same agent state, two processes.
|
||||
|
||||
[Unit]
|
||||
Description=Hermes web dashboard container
|
||||
|
||||
[Container]
|
||||
ContainerName=hermes-dashboard
|
||||
Image=localhost/hermes-agent:latest
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
Environment=HERMES_DATA_DIR=/data
|
||||
Environment=HERMES_HOME=/data
|
||||
Volume=/home/dank/Containers/hermes/data:/data
|
||||
PublishPort=9119:9119
|
||||
Exec=bash -c 'hermes dashboard --host 0.0.0.0 --port 9119 --no-open --skip-build'
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,65 +0,0 @@
|
||||
# Quadlet unit: Hermex webui (Hermex) container (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/hermes-webui.container
|
||||
# Unit name: hermes-webui.service (container name: hermes-webui)
|
||||
#
|
||||
# Runs the open-source hermes-webui (github.com/nesquena/hermes-webui) in
|
||||
# foreground mode: bootstrap.py creates .venv inside the mounted checkout,
|
||||
# installs WebUI + agent deps, then execs server.py as a single foreground
|
||||
# process. It reads HERMES_HOME=/data, so it drives the SAME agent state
|
||||
# (memory, skills, sessions) as the gateway container.
|
||||
#
|
||||
# Host networking on purpose: the agent's chrome-devtools MCP client spawns
|
||||
# npx chrome-devtools-mcp with --browserUrl=http://127.0.0.1:9222, and the CDP
|
||||
# daemon binds loopback-only in the host network namespace (see
|
||||
# chrome-devtools-protocol.container). Only processes sharing that namespace
|
||||
# can drive the browser; a bridge-network client could not reach it at all.
|
||||
# The server binds 0.0.0.0:8787 of the host netns, and that is deliberate:
|
||||
# this unit is host-networked (the CDP daemon above needs a shared loopback), so
|
||||
# Caddy cannot reach it by container name and fronts the vhost
|
||||
# hermes.franksdank.cloud with `reverse_proxy host.containers.internal:8787`
|
||||
# (netavark's host address, 169.254.1.2) — not the host LAN IP, which Caddy's
|
||||
# bridge netns cannot reach for a bare host-netns bind; the measured asymmetry
|
||||
# with published ports is recorded in the hermes block of dankclaw/Caddyfile.
|
||||
# A loopback bind was tried on 2026-09-16 for a tighter posture and broke that
|
||||
# vhost, because a bind is also the only way an off-host process can reach a
|
||||
# host-networked container: the LAN address and the tunnel's localhost hop are
|
||||
# both served by the wildcard, and a loopback-only bind serves only the latter.
|
||||
# Controls are the auth chain in front of the vhost plus the password.
|
||||
|
||||
[Unit]
|
||||
Description=Hermes webui (Hermex) container
|
||||
|
||||
[Container]
|
||||
ContainerName=hermes-webui
|
||||
Image=localhost/hermes-agent:latest
|
||||
Pull=never
|
||||
Network=host
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=HERMES_WEBUI_HOST=0.0.0.0
|
||||
Environment=HERMES_WEBUI_PORT=8787
|
||||
Environment=HERMES_WEBUI_PASSWORD=${HERMES_WEBUI_PASSWORD}
|
||||
Environment=HERMES_HOME=/data
|
||||
# Headless host: never show the first-run wizard (a profile already
|
||||
# exists in /data from the gateway).
|
||||
Environment=HERMES_WEBUI_SKIP_ONBOARDING=1
|
||||
Volume=/home/dank/Containers/hermes/data:/data
|
||||
Volume=/home/dank/Containers/hermes/webui:/webui
|
||||
Exec=python3 /webui/bootstrap.py --foreground
|
||||
# JSON exec form on purpose: this quadlet generator joins a bare HealthCmd
|
||||
# into ONE argv element with \x20 escapes and then shell-executes it — the
|
||||
# python one-liner's quoting would be mangled. A JSON array is passed through
|
||||
# verbatim. Start period covers first-boot pip installs into /webui/.venv.
|
||||
HealthCmd=["python3","-c","import urllib.request;urllib.request.urlopen('http://127.0.0.1:8787/health')"]
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=5
|
||||
HealthStartPeriod=600s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,45 +0,0 @@
|
||||
# Quadlet unit: Hermes agent gateway (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/hermes.container
|
||||
# Unit name: hermes.service (container name: hermes)
|
||||
#
|
||||
# The messaging gateway (Signal adapter -> the host signal-cli daemon). The
|
||||
# image's default s6 entrypoint would run the interactive TUI, which exits
|
||||
# without a TTY; Exec= below replaces it with the headless gateway command,
|
||||
# spliced by the v1 quadlet generator as trailing args after the image ref
|
||||
# (same splice pattern proven live on authentik-worker). The gateway drops to
|
||||
# the unprivileged hermes user (uid 10000) at runtime; /data is world-writable
|
||||
# on purpose so that unmapped uid can write agent.log and its state there.
|
||||
#
|
||||
# Host networking: the adapter dials the signal-cli daemon, a HOST process
|
||||
# listening on :8080. The compose era dialed the host LAN IP from a bridge
|
||||
# network, where only DNAT'd published ports accept connections to that
|
||||
# address — un-forwarded host ports refuse instantly (verified live
|
||||
# 2026-08-30). A host-network client dials 127.0.0.1:8080 with no such
|
||||
# dependency, and the gateway publishes nothing inbound.
|
||||
|
||||
[Unit]
|
||||
Description=Hermes agent gateway container
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
Network=host
|
||||
Image=localhost/hermes-agent:latest
|
||||
Pull=never
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=HERMES_DATA_DIR=/data
|
||||
Environment=HERMES_HOME=/data
|
||||
Environment=SIGNAL_HTTP_URL=http://127.0.0.1:8080
|
||||
Environment=SIGNAL_ACCOUNT=${SIGNAL_ACCOUNT}
|
||||
Environment=SIGNAL_ALLOWED_USERS=${SIGNAL_ALLOWED_USERS}
|
||||
Volume=/home/dank/Containers/hermes/data:/data
|
||||
Exec=bash -c 'hermes gateway run --force -v'
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,43 +0,0 @@
|
||||
# If you change this file or any /etc/default/grub.d/*.cfg file,
|
||||
# run 'update-grub' afterwards to update /boot/grub/grub.cfg.
|
||||
# For full documentation of the options in these files, see:
|
||||
# info -f grub -n 'Simple configuration'
|
||||
|
||||
GRUB_DEFAULT=0
|
||||
GRUB_TIMEOUT_STYLE=hidden
|
||||
GRUB_TIMEOUT=0
|
||||
GRUB_DISTRIBUTOR=`( . /etc/os-release && echo ${NAME} )`
|
||||
# Identity (passthrough) DMA since 2026-10-18: the IOMMU stays enabled, so the
|
||||
# XDNA2 NPU (/dev/accel/accel0, PASID/SVA) is usable again, while the iGPU's DMA
|
||||
# runs untranslated. Decided in serverAppConfigs-docs/LLM/
|
||||
# dankclaw-iommu-translation-off.md; install with scripts/set-dankclaw-bootline.sh.
|
||||
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=pt"
|
||||
|
||||
# If your computer has multiple operating systems installed, then you
|
||||
# probably want to run os-prober. However, if your computer is a host
|
||||
# for guest OSes installed via LVM or raw disk devices, running
|
||||
# os-prober can cause damage to those guest OSes as it mounts
|
||||
# filesystems to look for things.
|
||||
#GRUB_DISABLE_OS_PROBER=false
|
||||
|
||||
# Uncomment to enable BadRAM filtering, modify to suit your needs
|
||||
# This works with Linux (no patch required) and with any kernel that obtains
|
||||
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
|
||||
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
|
||||
|
||||
# Uncomment to disable graphical terminal
|
||||
#GRUB_TERMINAL=console
|
||||
|
||||
# The resolution used on graphical terminal
|
||||
# note that you can use only modes which your graphic card supports via VBE/GOP/UGA
|
||||
# you can see them in real GRUB with the command `videoinfo'
|
||||
#GRUB_GFXMODE=640x480
|
||||
|
||||
# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
|
||||
#GRUB_DISABLE_LINUX_UUID=true
|
||||
|
||||
# Uncomment to disable generation of recovery mode menu entries
|
||||
#GRUB_DISABLE_RECOVERY="true"
|
||||
|
||||
# Uncomment to get a beep at grub start
|
||||
#GRUB_INIT_TUNE="480 440 1"
|
||||
@@ -1,65 +0,0 @@
|
||||
#!/bin/sh
|
||||
# halogen-fabric-clock hold | release | status: the AMD GPU's fabric clock (FCLK), held at its top level.
|
||||
#
|
||||
# On Strix Halo the GPU and the NPU working at the same time can hang the machine, and corrupt the NPU's results,
|
||||
# while the GPU's fabric clock changes speed; held at its top level, the two run together cleanly. halogen-flash-server
|
||||
# with HALOGEN_NPU_MODELS refuses to start until the clock is held. This holds it: the performance level `manual` and
|
||||
# only the top pp_dpm_fclk level, on every amdgpu device that has the control (the GPU and SoC clocks stay automatic).
|
||||
# Cost: about 1 W at idle. Run as root; halogen-fabric-clock.service runs `hold` at every boot and after a resume.
|
||||
#
|
||||
# hold hold it (waits up to 60 s for the GPU driver to come up)
|
||||
# release give it back to the driver (the performance level `auto`)
|
||||
# status print each device's state; exit 0 when every device is held
|
||||
set -u
|
||||
SYS="${HALOGEN_SYSFS:-/sys}"
|
||||
|
||||
devices() {
|
||||
for d in "$SYS"/class/drm/card*/device; do
|
||||
[ -w "$d/pp_dpm_fclk" ] || [ -r "$d/pp_dpm_fclk" ] || continue
|
||||
[ -e "$d/power_dpm_force_performance_level" ] || continue
|
||||
echo "$d"
|
||||
done | sort -u
|
||||
}
|
||||
|
||||
held() { # held DEVICE_DIR: the performance level high, or manual with only the top fclk level selected
|
||||
perf=$(cat "$1/power_dpm_force_performance_level" 2>/dev/null)
|
||||
[ "$perf" = high ] && return 0
|
||||
[ "$perf" = manual ] || return 1
|
||||
top=$(grep . "$1/pp_dpm_fclk" | tail -n 1)
|
||||
[ "$(grep -c '\*' "$1/pp_dpm_fclk")" = 1 ] && [ "${top%\*}" != "$top" ]
|
||||
}
|
||||
|
||||
cmd="${1:-status}"
|
||||
case "$cmd" in
|
||||
hold)
|
||||
n=0
|
||||
while [ -z "$(devices)" ] && [ "$n" -lt 60 ]; do sleep 1; n=$((n + 1)); done
|
||||
[ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control (pp_dpm_fclk) under $SYS/class/drm" >&2; exit 1; }
|
||||
rc=0
|
||||
for d in $(devices); do
|
||||
lvl=$(grep . "$d/pp_dpm_fclk" | tail -n 1 | cut -d: -f1)
|
||||
# the driver applies the level several seconds after the write, so the read-back waits up to 30 s
|
||||
ok=0
|
||||
if echo manual > "$d/power_dpm_force_performance_level" && echo "$lvl" > "$d/pp_dpm_fclk"; then
|
||||
t=0; until held "$d" || [ "$t" -ge 300 ]; do sleep 0.1; t=$((t + 1)); done
|
||||
held "$d" && ok=1
|
||||
fi
|
||||
if [ "$ok" = 1 ]; then
|
||||
echo "halogen-fabric-clock: $d: held at level $lvl ($(grep . "$d/pp_dpm_fclk" | tail -n 1 | awk '{print $2}'))"
|
||||
else
|
||||
echo "halogen-fabric-clock: $d: could not hold it (level $lvl)" >&2; rc=1
|
||||
fi
|
||||
done
|
||||
exit "$rc" ;;
|
||||
release)
|
||||
for d in $(devices); do echo auto > "$d/power_dpm_force_performance_level" && echo "halogen-fabric-clock: $d: released (auto)"; done ;;
|
||||
status)
|
||||
[ -n "$(devices)" ] || { echo "halogen-fabric-clock: no amdgpu device with a fabric clock control"; exit 1; }
|
||||
rc=0
|
||||
for d in $(devices); do
|
||||
if held "$d"; then echo "$d: held ($(cat "$d/power_dpm_force_performance_level"))"
|
||||
else echo "$d: NOT held ($(cat "$d/power_dpm_force_performance_level"))"; rc=1; fi
|
||||
done
|
||||
exit "$rc" ;;
|
||||
*) echo "usage: halogen-fabric-clock hold | release | status" >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -1,19 +0,0 @@
|
||||
# Holds the AMD GPU's fabric clock at its top level at boot and after every resume, for halogen-flash-server with
|
||||
# the NPU (HALOGEN_NPU_MODELS): on Strix Halo the GPU and the NPU at once are safe only with that clock held.
|
||||
# Install (as root), from this directory:
|
||||
# install -m 755 halogen-fabric-clock /usr/local/sbin/
|
||||
# install -m 644 halogen-fabric-clock.service /etc/systemd/system/
|
||||
# systemctl daemon-reload && systemctl enable --now halogen-fabric-clock.service
|
||||
# Check: halogen-fabric-clock status. Undo: systemctl disable halogen-fabric-clock.service, then
|
||||
# halogen-fabric-clock release (or reboot). The unit runs once at boot and once after each resume (a resume can hand
|
||||
# the clock back to the driver); it holds nothing open, so it shows inactive between runs.
|
||||
[Unit]
|
||||
Description=Hold the AMD GPU fabric clock at its top level (GPU and NPU together)
|
||||
After=systemd-modules-load.service suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/halogen-fabric-clock hold
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target suspend.target hibernate.target hybrid-sleep.target suspend-then-hibernate.target
|
||||
@@ -1,30 +0,0 @@
|
||||
# Loki + a static busybox.
|
||||
#
|
||||
# grafana/loki is distroless: no shell, no wget, so nothing can run inside the
|
||||
# container to check it. This image drops a static busybox into it for the
|
||||
# healthcheck. All Loki behavior is unchanged.
|
||||
#
|
||||
# The healthcheck lives HERE, in exec form, and not in the Quadlet unit: a
|
||||
# unit's HealthCmd reaches podman as CMD-SHELL, which needs /bin/sh — the one
|
||||
# thing this image does not have. Specifying it here is also what lets the
|
||||
# container report its own health instead of looking merely "Up" while wedged.
|
||||
#
|
||||
# Build on dankclaw:
|
||||
# podman build --format docker -t franksdank/loki:latest \
|
||||
# -f dankclaw/loki/Containerfile .
|
||||
#
|
||||
# --format docker is required: podman's default OCI image format has no
|
||||
# HEALTHCHECK field, so a default-format build warns and silently discards the
|
||||
# one below, leaving a container that reports no health at all.
|
||||
|
||||
FROM docker.io/library/alpine:latest AS tools
|
||||
# busybox-static is statically linked: the target images have no dynamic
|
||||
# loader (loki is distroless), so the alpine default /bin/busybox (musl
|
||||
# dynamic) would fail with "no such file or directory".
|
||||
RUN apk add --no-cache busybox-static
|
||||
|
||||
FROM docker.io/grafana/loki:latest
|
||||
COPY --from=tools /bin/busybox.static /usr/local/bin/busybox
|
||||
# Exec form (JSON array): no shell is involved.
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
|
||||
CMD ["/usr/local/bin/busybox", "wget", "--spider", "-q", "http://localhost:3100/ready"]
|
||||
@@ -1,34 +0,0 @@
|
||||
# Quadlet unit: Marreta browser companion — Lightpanda (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/marreta-browser.container
|
||||
# Unit name: marreta-browser.service (container name: marreta_browser)
|
||||
#
|
||||
# Lightpanda is a headless browser that exposes the Chrome DevTools Protocol
|
||||
# over a WebSocket on port 9222. Marreta drives it to render JS-heavy pages
|
||||
# the plain curl fetch cannot. It shares gluetun's network namespace so the
|
||||
# browser's OWN outbound fetches also exit the WireGuard tunnel — this is
|
||||
# required because marreta's PROXY_URL only covers the curl strategy; the
|
||||
# CDP session passes no proxy to the browser, so without netns sharing the
|
||||
# browser would leak the host's real egress IP.
|
||||
#
|
||||
# Marreta reaches it on loopback (same netns): ws://127.0.0.1:9222. No
|
||||
# published port — nothing outside the netns talks to it directly.
|
||||
# See serverAppConfigs-docs/Dankclaw/Marreta.md.
|
||||
|
||||
[Unit]
|
||||
Description=Marreta Lightpanda browser companion container
|
||||
Wants=gluetun.service
|
||||
After=gluetun.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=marreta_browser
|
||||
Network=gluetun.container
|
||||
Image=docker.io/lightpanda/browser:nightly
|
||||
|
||||
[Service]
|
||||
TimeoutStartSec=300
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,66 +0,0 @@
|
||||
# Quadlet unit: Marreta content-cleaning proxy (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/marreta.container
|
||||
# Unit name: marreta.service (container name: marreta)
|
||||
#
|
||||
# Marreta is a Laravel app (serversideup/php:8.4-fpm-nginx) that strips
|
||||
# paywalls/tracking from a fetched page. It reuses gluetun's network
|
||||
# namespace (Network=gluetun.container) so ALL of its outbound fetches exit
|
||||
# the same WireGuard tunnel as SearXNG — the curl strategy AND, because the
|
||||
# browser companion shares the same netns, the Lightpanda fetches too
|
||||
# (marreta's PROXY_URL does NOT cover the browser: the CDP connection passes
|
||||
# no proxy, so netns sharing is the only way to keep the browser's egress on
|
||||
# the tunnel).
|
||||
#
|
||||
# Port: the shared netns already has SearXNG on 8080, so marreta must NOT
|
||||
# bind 8080. NGINX_HTTP_PORT=8085 relocates it (the fpm-nginx image does
|
||||
# not bake conf.d/*.conf; the site config is envsubst'd from
|
||||
# ${NGINX_HTTP_PORT} at container start). Gluetun publishes host 8082 ->
|
||||
# 8085 so Caddy can reach it; see dankclaw/searxng/quadlet/gluetun.container.
|
||||
# The app reaches its browser companion over loopback (same netns):
|
||||
# ws://127.0.0.1:9222.
|
||||
#
|
||||
# Secrets (APP_KEY, admin creds) live in /home/dank/Containers/marreta/env
|
||||
# (mode 0600, dank-owned so the rootless unit can read it); this file holds
|
||||
# none. See serverAppConfigs-docs/Dankclaw/Marreta.md.
|
||||
|
||||
[Unit]
|
||||
Description=Marreta content-cleaning proxy container
|
||||
Wants=gluetun.service marreta-browser.service
|
||||
After=gluetun.service marreta-browser.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=marreta
|
||||
Network=gluetun.container
|
||||
Image=ghcr.io/manualdousuario/marreta:latest
|
||||
# Relocate off SearXNG's 8080 inside the shared netns.
|
||||
Environment=NGINX_HTTP_PORT=8085
|
||||
Environment=APP_NAME=Marreta
|
||||
Environment=APP_ENV=production
|
||||
Environment=APP_DEBUG=false
|
||||
Environment=APP_URL=https://marreta.franksdank.cloud
|
||||
Environment=APP_LOCALE=en
|
||||
Environment="APP_DESCRIPTION=Paywall-free reading"
|
||||
Environment=BROWSER_WS_ENDPOINT=ws://127.0.0.1:9222
|
||||
Environment=DISABLE_CACHE=false
|
||||
# Secrets expanded from the EnvironmentFile below.
|
||||
Environment=APP_KEY=${MARRETA_APP_KEY}
|
||||
Environment=ADMIN_EMAIL=${MARRETA_ADMIN_EMAIL}
|
||||
Environment=ADMIN_PASSWORD=${MARRETA_ADMIN_PASSWORD}
|
||||
# Named volume: podman auto-creates it and copyups the image's storage/app
|
||||
# ownership (www-data), so the app can write its SQLite DB + cache.
|
||||
Volume=marreta_storage:/var/www/html/storage/app
|
||||
HealthCmd=["curl","-fsS","-o","/dev/null","http://127.0.0.1:8085/up"]
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=5
|
||||
HealthStartPeriod=120s
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/marreta/env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,54 +0,0 @@
|
||||
# Quadlet unit: alertmanager (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/alertmanager.container
|
||||
# Unit name: alertmanager.service (container name: alertmanager)
|
||||
#
|
||||
# The delivery half of the LLM alert rules. Until 2026-09-17 the rules in
|
||||
# shared/prometheus/rules/llm-fleet.yml evaluated and reached nobody — a
|
||||
# firing alert was visible in Prometheus and on the Grafana dashboard and
|
||||
# nowhere else, and an undelivered alert looks identical to a quiet one.
|
||||
# This unit closes that gap: Prometheus pushes to alertmanager:9093 over the
|
||||
# dankclaw_default network, and the single receiver webhooks the host-side
|
||||
# signal-relay user unit, which hands the text to signal-cli under Frank's
|
||||
# existing Signal account.
|
||||
#
|
||||
# PUBLISHING. 127.0.0.1 only, like every other observability unit: the
|
||||
# unauthenticated Alertmanager API (lifecycle endpoints are always on in
|
||||
# this binary — there is no flag to gate them) has no business on a host
|
||||
# interface, and no off-host consumer needs it. Caddy gains no vhost for
|
||||
# it; the homepage timer probes loopback.
|
||||
#
|
||||
# CONFIG. /opt/alertmanager/config/alertmanager.yml is installed from
|
||||
# shared/prometheus/alertmanager.yml by deploy-remote.sh on every deploy, so
|
||||
# the repo copy is the source of truth. STATE lives in /opt/alertmanager/data.
|
||||
#
|
||||
# Rootless: container root maps to the invoking user (dank) in this user
|
||||
# namespace and dank owns /opt/alertmanager/data. The image's default user
|
||||
# (nobody) maps to a subuid that cannot write it — same User=0:0 as prometheus.
|
||||
|
||||
[Unit]
|
||||
Description=Alertmanager alert delivery (Dankclaw)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=alertmanager
|
||||
Image=docker.io/prom/alertmanager:latest
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
|
||||
PublishPort=127.0.0.1:9093:9093
|
||||
Volume=/opt/alertmanager/config:/etc/alertmanager
|
||||
Volume=/opt/alertmanager/data:/alertmanager
|
||||
HealthCmd=wget --spider -q http://localhost:9093/-/healthy
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,59 +0,0 @@
|
||||
# Quadlet unit: alloy (ROOT system service, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: /etc/containers/systemd/alloy.container
|
||||
# Unit name: alloy.service (container name: alloy) — SYSTEM manager, root.
|
||||
#
|
||||
# WHY ROOT (not rootless, unlike its observability siblings). Alloy reads the
|
||||
# host journal. The journal files are root:systemd-journal 0640 (other::---);
|
||||
# a rootless container carries no supplementary groups and cannot open them —
|
||||
# a rootless read of system.journal returns DENIED (see the Alloy page in the
|
||||
# vault). Running as root, one loki.source.journal reads system.journal AND
|
||||
# user-1000.journal, so it carries both host logs and the rootless Quadlet
|
||||
# containers' logs. Validated 2026-10-20: a root probe read 31,825 journal
|
||||
# lines and pushed 26,966 into Loki in 20 s, zero drops.
|
||||
#
|
||||
# NETWORK=host. Loki publishes 127.0.0.1:3100 (loopback only) on the host, and
|
||||
# no bridge container — rootless or root — can reach the host's loopback, so
|
||||
# this unit runs on the host network and pushes to 127.0.0.1:3100. The side
|
||||
# effect: the UI must bind loopback too, so Exec sets the listen-addr to
|
||||
# 127.0.0.1 (the old rootless unit used 0.0.0.0 only because container loopback
|
||||
# was not host loopback; under Network=host they are the same address, so the
|
||||
# unauthenticated UI stays loopback-only — nothing off-host reaches it).
|
||||
#
|
||||
# THE /etc/machine-id MOUNT IS LOAD-BEARING. sd_journal locates the journal
|
||||
# directory by the local machine id; without it the container opens nothing and
|
||||
# the journal source reads exactly 0 lines — silently, no error, health green.
|
||||
# This mount is the entire reason the pipeline carries data. Never remove it.
|
||||
#
|
||||
# IMAGE. Built rootless by build_compose_image alloy, then mirrored into the
|
||||
# root image store by the deployer (a root build runs in the build container's
|
||||
# own resolver and fails on DNS here). Pull=never: boot must not reach for the
|
||||
# local tag at a registry.
|
||||
|
||||
[Unit]
|
||||
Description=Alloy log shipper (Dankclaw)
|
||||
After=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=alloy
|
||||
Image=localhost/franksdank/alloy:latest
|
||||
Pull=never
|
||||
Network=host
|
||||
Exec=run --server.http.listen-addr=127.0.0.1:12345 /etc/alloy/config.alloy
|
||||
Volume=/opt/alloy/config:/etc/alloy:ro
|
||||
Volume=/etc/machine-id:/etc/machine-id:ro
|
||||
Volume=/run/systemd/journal/socket:/run/systemd/journal/socket:ro
|
||||
Volume=/var/log/journal:/var/log/journal:ro
|
||||
Volume=/run/log/journal:/run/log/journal:ro
|
||||
HealthCmd=/usr/local/bin/busybox wget --spider -q http://127.0.0.1:12345/-/healthy
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,56 +0,0 @@
|
||||
# Quadlet unit: cadvisor (rootless, boot-persistent) — DANKCLAW.
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/cadvisor.container
|
||||
# Unit name: cadvisor.service (container name: cadvisor)
|
||||
#
|
||||
# WHY THIS EXISTS. The app stack is ~34 rootless Podman containers and until
|
||||
# now nothing reported per-container CPU/memory — Grafana could only see the
|
||||
# host (node_exporter) and the LLM engines. cadvisor reads the host cgroup
|
||||
# tree and exports container_cpu_* / container_memory_* per container.
|
||||
#
|
||||
# THE LOAD-BEARING FLAG IS --cgroupns=host. Podman starts every container in a
|
||||
# PRIVATE cgroup namespace, so an exporter container sees only its OWN cgroup
|
||||
# subtree: cadvisor exported a single root series until this flag was added,
|
||||
# then it walked the whole tree (996 per-container series measured 2026-10-20).
|
||||
# The quadlet unit name is carried in the cgroup path — app.slice/<name>.service/
|
||||
# libpod-payload-<hex> — so Prometheus relabels that path into a `container`
|
||||
# label (shared/prometheus/prometheus.yml, cadvisor-dankclaw job) and no
|
||||
# libpod/podman socket is needed to name containers.
|
||||
#
|
||||
# BINDING. cadvisor has no per-interface listen flag (unlike node_exporter's
|
||||
# --web.listen-address), so it binds 0.0.0.0 INSIDE its own network namespace.
|
||||
# The bridge PublishPort is what keeps the unauthenticated endpoint off the
|
||||
# host's other interfaces: it publishes exactly loopback (host-local probes and
|
||||
# the deployer's health gate) and this host's LAN address (Prometheus, a bridge
|
||||
# container, reaches a host-published port at its LAN address — the same netavark
|
||||
# rule node-exporter relies on, only here the port really is published).
|
||||
#
|
||||
# It is rootless on purpose: dank owns the app containers' cgroups, so a
|
||||
# rootless cadvisor reads every one of them; root would buy nothing here.
|
||||
|
||||
[Unit]
|
||||
Description=cAdvisor container metrics (Dankclaw)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=cadvisor
|
||||
Image=gcr.io/cadvisor/cadvisor:v0.49.1
|
||||
PodmanArgs=--pid=host --cgroupns=host
|
||||
PublishPort=127.0.0.1:8080:8080
|
||||
PublishPort=192.168.0.143:8080:8080
|
||||
Volume=/sys:/sys:ro
|
||||
Volume=/:/rootfs:ro
|
||||
Exec=--housekeeping_interval=30s
|
||||
HealthCmd=wget --spider -q http://localhost:8080/healthz
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=40s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,94 +0,0 @@
|
||||
# Quadlet unit: grafana (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/grafana.container
|
||||
# Unit name: grafana.service (container name: grafana)
|
||||
#
|
||||
# Dashboards. This replaces the compose service of the same name in
|
||||
# dankclaw/compose.yaml ("Observability" block).
|
||||
#
|
||||
# PUBLISHING. Bound to 127.0.0.1 for the same reason as prometheus: the browser
|
||||
# path is https://grafana.franksdank.cloud (Caddy → Anubis → Authentik outpost →
|
||||
# grafana:3000 over dankclaw_default) and the host-side homepage status timer
|
||||
# probes 127.0.0.1:3002. Publishing on 0.0.0.0 as the compose service did left
|
||||
# an off-host path into Grafana that skips the guard chain entirely.
|
||||
#
|
||||
# AUTH. Grafana is an OAuth2 relying party against Authentik (OAuth2Provider
|
||||
# "grafana", confidential client). The guard chain fronts unauthenticated
|
||||
# traffic; the login itself happens through the Authentik button inside
|
||||
# Grafana. The client secret is NOT written here: it is expanded by systemd
|
||||
# from the root .env (EnvironmentFile in [Service], never in [Container], so
|
||||
# the container receives exactly this one secret and not the whole file).
|
||||
# Without that .env entry the unit will not start — the same dependency every
|
||||
# other rendered-secret deploy has.
|
||||
#
|
||||
# STATE. /opt/grafana/data holds grafana.db. /opt/grafana/provisioning is
|
||||
# mounted read-only and was EMPTY as of 2026-09-11, and grafana.db reported
|
||||
# zero datasources and zero dashboards: this Grafana is an empty install whose
|
||||
# only configured account is the local admin. Nothing here queries Prometheus
|
||||
# or Loki yet.
|
||||
|
||||
[Unit]
|
||||
Description=Grafana dashboards (Dankclaw)
|
||||
After=network-online.target prometheus.service
|
||||
Wants=network-online.target prometheus.service
|
||||
|
||||
[Container]
|
||||
ContainerName=grafana
|
||||
Image=docker.io/grafana/grafana:latest
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
PublishPort=127.0.0.1:3002:3000
|
||||
Environment=GF_SERVER_ROOT_URL=https://grafana.franksdank.cloud
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_ENABLED=true
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_NAME=Authentik
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_ID=grafana-21ca190cde75d60f
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=${GRAFANA_OAUTH_CLIENT_SECRET}
|
||||
# Global authorize path: this Authentik build resolves the provider by the
|
||||
# client_id query parameter — /application/o/<slug>/authorize/ does not exist.
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_AUTH_URL=https://auth.franksdank.cloud/application/o/authorize/
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_TOKEN_URL=https://auth.franksdank.cloud/application/o/token/
|
||||
# Standard OIDC userinfo. Grafana v13 fetches <api_url>/emails for the address
|
||||
# ONLY when the token carries no `email` claim (authentik has no /emails path and
|
||||
# 404s it, failing the login) — so the claim must actually be there: the fix is
|
||||
# the grafana-email-scope mapping (see scripts/authentik-grafana-sso-fix.sh),
|
||||
# not a Grafana flag. Do NOT append ?api=1 to this URL: Grafana then GETs
|
||||
# <...>/userinfo/?api=1/emails, a 403.
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_API_URL=https://auth.franksdank.cloud/application/o/userinfo/
|
||||
# authentik's scopes are openid/email/profile. `user:email` (a GitLab-ism) is
|
||||
# not a known scope here; requesting it makes the authorize view crash.
|
||||
# Quoted: quadlet word-splits Environment=, so an unquoted spaced list
|
||||
# truncates SCOPES to its first token.
|
||||
Environment="GF_AUTH_GENERIC_OAUTH_SCOPES=openid email profile"
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_EMAIL_ATTRIBUTE_PATH=email
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_LOGIN_ATTRIBUTE_PATH=sub
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_NAME_ATTRIBUTE_PATH=preferred_username
|
||||
# First SSO login auto-provisions the user. Safe: unauthenticated traffic never
|
||||
# reaches this callback — the Anubis -> Authentik guard fronts every request.
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=true
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_AUTO_UPSERT=true
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_ALLOW_ASSIGN_ORG=true
|
||||
Environment=GF_AUTH_GENERIC_OAUTH_ORG_ID=1
|
||||
Volume=/opt/grafana/data:/var/lib/grafana
|
||||
Volume=/opt/grafana/provisioning:/etc/grafana/provisioning:ro
|
||||
HealthCmd=wget --spider -q http://localhost:3000/api/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=60s
|
||||
|
||||
[Service]
|
||||
# systemd-side only: supplies ${GRAFANA_OAUTH_CLIENT_SECRET} for the expansion
|
||||
# above. The container never receives this file.
|
||||
#
|
||||
# It is the per-service env file the deployer seeds from dankclaw/.env — the
|
||||
# fleet convention (caddy, pihole, copyparty, romm and seerr all read one).
|
||||
# The repo-root .env that the compose service happened to pick up is a gluetun
|
||||
# env file: it carries HOME=/root and the WireGuard keys, and importing it into
|
||||
# this service makes podman look for its configuration under /root, which it
|
||||
# cannot read, so the unit fails to start. Seed the value into dankclaw/.env.
|
||||
EnvironmentFile=/home/dank/Containers/grafana/env
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,57 +0,0 @@
|
||||
# Quadlet unit: loki (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/loki.container
|
||||
# Unit name: loki.service (container name: loki)
|
||||
#
|
||||
# Log store. This replaces the compose service of the same name in
|
||||
# dankclaw/compose.yaml ("Observability" block).
|
||||
#
|
||||
# IMAGE. franksdank/loki is built locally by deploy-remote.sh
|
||||
# (build_compose_image) from dankclaw/loki/Containerfile: upstream
|
||||
# grafana/loki plus a static busybox, because loki is distroless and the
|
||||
# healthcheck has nothing to run inside it. Pull=never keeps a boot from
|
||||
# trying to fetch that local tag from a registry. The localhost/ prefix is
|
||||
# required — this host's quadlet path cannot resolve a registry-prefixed ref
|
||||
# from local storage at create time.
|
||||
#
|
||||
# PUBLISHING. Bound to 127.0.0.1: Alloy pushes to loki:3100 over
|
||||
# dankclaw_default, Caddy proxies the guarded vhost to loki:3100 the same way,
|
||||
# and the homepage status timer probes 127.0.0.1:3100. Loki has
|
||||
# auth_enabled: false, so the compose service's 0.0.0.0 binding was an
|
||||
# unauthenticated log-ingest and read API on every host interface.
|
||||
#
|
||||
# CONFIG. /opt/loki/config/loki-config.yaml is installed from
|
||||
# shared/loki/loki-config.yaml on every deploy (single-binary, filesystem
|
||||
# storage, 30-day retention). Chunks and rules live in /opt/loki/data.
|
||||
#
|
||||
# Rootless: container root maps to the invoking user (dank), who owns
|
||||
# /opt/loki/data; the image's default user (10001) maps to a subuid that
|
||||
# cannot write it.
|
||||
|
||||
[Unit]
|
||||
Description=Loki log store (Dankclaw)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=loki
|
||||
Image=localhost/franksdank/loki:latest
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
Exec=-config.file=/etc/loki/loki-config.yaml
|
||||
PublishPort=127.0.0.1:3100:3100
|
||||
Volume=/opt/loki/config:/etc/loki
|
||||
Volume=/opt/loki/data:/loki
|
||||
# No HealthCmd here on purpose. A unit's HealthCmd reaches podman as
|
||||
# CMD-SHELL, which means `/bin/sh -c <command>` — and this image is
|
||||
# distroless, with no shell at all. The healthcheck therefore ships in the
|
||||
# image (dankclaw/loki/Containerfile) in exec form, where it also carries its
|
||||
# interval, timeout, start period and retry count.
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,54 +0,0 @@
|
||||
# Quadlet unit: node-exporter (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
|
||||
# Unit name: node-exporter.service (container name: node-exporter)
|
||||
#
|
||||
# Host metrics source for Prometheus. This replaces the compose service of the
|
||||
# same name in dankclaw/compose.yaml ("Observability" block).
|
||||
#
|
||||
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||
# host's own interfaces/connections, and --pid=host for the processes and
|
||||
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||
# loopback, which is what the homepage status timer probes.
|
||||
#
|
||||
# LISTEN ADDRESSES. Until 2026-09-11 this bound :9100 on every interface. It
|
||||
# now binds exactly two: loopback for the homepage status probe, and the LAN
|
||||
# address because Prometheus — which runs in a container on dankclaw_default —
|
||||
# scrapes http://192.168.0.143:9100 (shared/prometheus/prometheus.yml,
|
||||
# job "node-dankclaw"). node_exporter itself has no authentication, so keeping
|
||||
# the wildcard bind off the table matters: this is the second half of the
|
||||
# pairing that removes the unauthenticated observability surface, and both
|
||||
# consumers are accounted for. Dankserver runs its own copy of this container
|
||||
# (scraped by the "node-dankserver" job) — since 2026-09-16 as a Quadlet unit in
|
||||
# the same shape (dankserver/observability/quadlet/), because the compose-only
|
||||
# version it replaced had been dead for a day and a half before anyone noticed.
|
||||
#
|
||||
# MOUNTS. The container reads the host's /proc, /sys and / as /host/* read-only
|
||||
# and is pointed at them with --path.*; it writes nothing anywhere.
|
||||
|
||||
[Unit]
|
||||
Description=Prometheus node exporter (Dankclaw host metrics)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=node-exporter
|
||||
Image=docker.io/prom/node-exporter:latest
|
||||
Network=host
|
||||
PodmanArgs=--pid=host
|
||||
Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.143:9100
|
||||
Volume=/proc:/host/proc:ro
|
||||
Volume=/sys:/host/sys:ro
|
||||
Volume=/:/rootfs:ro
|
||||
HealthCmd=wget --spider -q http://localhost:9100/metrics
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,64 +0,0 @@
|
||||
# Quadlet unit: prometheus (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/prometheus.container
|
||||
# Unit name: prometheus.service (container name: prometheus)
|
||||
#
|
||||
# The homelab metrics store. This replaces the compose service of the same
|
||||
# name in dankclaw/compose.yaml ("Observability" block): these five containers
|
||||
# were started by hand with podman-compose and never came back after a reboot,
|
||||
# which the 2026-09-11 reboot for the amd_iommu=off change made obvious.
|
||||
#
|
||||
# PUBLISHING. The published port is bound to 127.0.0.1 deliberately. Prometheus
|
||||
# is reached by Caddy over the dankclaw_default network (the Caddyfile's
|
||||
# post-auth handler proxies to prometheus:9090) and by the host-side homepage
|
||||
# status timer over 127.0.0.1. The compose service published on 0.0.0.0, which
|
||||
# put an UNAUTHENTICATED metrics API — with the lifecycle endpoint enabled —
|
||||
# on every host interface: a bypass around the Anubis + Authentik guard on
|
||||
# prometheus.franksdank.cloud. No off-host consumer needs it.
|
||||
#
|
||||
# CONFIG. /opt/prometheus/config/prometheus.yml is installed from
|
||||
# shared/prometheus/prometheus.yml by deploy-remote.sh on every deploy, so the
|
||||
# repo copy is the source of truth. STATE (30-day retention) lives in
|
||||
# /opt/prometheus/data.
|
||||
#
|
||||
# Rootless: container root maps to the invoking user (dank) in this user
|
||||
# namespace and dank owns /opt/prometheus/data. The image's default user
|
||||
# (nobody) maps to a subuid that cannot write it.
|
||||
#
|
||||
# RULES. Five LLM alert rules and one recording rule load from
|
||||
# /etc/prometheus/rules/ (installed from shared/prometheus/rules/ by the
|
||||
# deployer). Delivery went live 2026-09-17: the alerting block in
|
||||
# prometheus.yml points at alertmanager:9093, which webhooks every firing
|
||||
# group to the host-side Signal relay (192.168.0.143:18093, reached from
|
||||
# the container as host.containers.internal), which sends through signal-cli account 318311.
|
||||
# The Grafana dashboard's ALERTS panel remains the second surface.
|
||||
#
|
||||
# The config volume is also why rules belong in git: editing them in the UI is
|
||||
# lost on the next deploy, since the deployer overwrites the whole directory.
|
||||
|
||||
[Unit]
|
||||
Description=Prometheus metrics store (Dankclaw)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=prometheus
|
||||
Image=docker.io/prom/prometheus:latest
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
Exec=--config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --storage.tsdb.retention.time=30d --web.enable-lifecycle
|
||||
PublishPort=127.0.0.1:9090:9090
|
||||
Volume=/opt/prometheus/config:/etc/prometheus
|
||||
Volume=/opt/prometheus/data:/prometheus
|
||||
HealthCmd=wget --spider -q http://localhost:9090/-/healthy
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,93 +0,0 @@
|
||||
# Quadlet unit: open-notebook's SurrealDB (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook-surrealdb.container
|
||||
# Unit name: open-notebook-surrealdb.service (container name: open-notebook-surrealdb)
|
||||
#
|
||||
# WHAT IT IS. open-notebook's only datastore: notebooks, notes, sources,
|
||||
# provider credentials and podcast episodes all live here. The app talks to it
|
||||
# over the WebSocket RPC on the private bridge (SURREAL_URL in
|
||||
# scripts/open-notebook.env), which is why the container NAME is part of the
|
||||
# contract — renaming it breaks the app's database URL, not just this file.
|
||||
#
|
||||
# IMAGE. `surrealdb/surrealdb:v2` pinned by manifest-list digest, resolved from
|
||||
# the registry on 2026-10-18 (that digest is SurrealDB 2.7.0, image built
|
||||
# 2026-09-23; `v2` itself is a MOVING tag, so the digest — not the tag — is what
|
||||
# makes this reproducible). Upstream open-notebook's own compose file pins the
|
||||
# same `v2` tag, and its Dockerfile copies /surreal out of the same image for
|
||||
# the retired single-container variant, so v2 is the version this app is built
|
||||
# against. Pull=never + the documented pull step (docs: [[Open Notebook]]):
|
||||
#
|
||||
# podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||
#
|
||||
# NO PUBLISHED PORT. 8000 is deliberately not published: the app reaches it by
|
||||
# name on systemd-open-notebook, and a published 8000 would put the RPC with
|
||||
# the root credentials on every interface (upstream publishes 127.0.0.1:8000
|
||||
# "purely for local debugging" — a `podman exec` probe is enough here). The
|
||||
# binary binds 0.0.0.0:8000 INSIDE its own namespace via the image's
|
||||
# SURREAL_BIND env; without that image default it would bind loopback only and
|
||||
# the app could never connect, so no --bind flag belongs here.
|
||||
#
|
||||
# CREDENTIALS, AND WHY THEY ARE NOT IN THE COMMAND LINE. The `surreal start`
|
||||
# CLI reads its options from the environment (`--username` is [env:
|
||||
# SURREAL_USER=], `--password` is [env: SURREAL_PASS=] — verified against the
|
||||
# pinned image's own `start --help`), so the credentials arrive through the two
|
||||
# EnvironmentFile lines and the argv stays clean. Passing `--pass` instead —
|
||||
# what upstream's compose does — would put the database password in
|
||||
# /proc/<pid>/cmdline, readable by any `ps` on the host. Note the two
|
||||
# spellings: the DATABASE reads SURREAL_PASS, the APP reads SURREAL_PASSWORD,
|
||||
# and scripts/open-notebook.env declares both because they must hold the same
|
||||
# value; the deployer writes that pair when it seeds the host secret file.
|
||||
#
|
||||
# NO USER= OVERRIDE. The image runs as uid 65532 (Chainguard glibc-dynamic
|
||||
# base). With a NAMED volume that works unmodified: podman creates the volume's
|
||||
# backing directory owned by the container's user (measured with this digest —
|
||||
# the host directory lands on the mapped subuid and the store starts clean),
|
||||
# which is exactly why upstream's `user: root` line is not repeated here: that
|
||||
# line is commented in upstream as being "required for bind mounts", and this
|
||||
# unit has none. The consequence is that the DB files are NOT readable by dank
|
||||
# without `podman unshare` — accepted for a datastore nobody edits by hand;
|
||||
# episode AUDIO is a host directory for the opposite reason (see the app unit).
|
||||
#
|
||||
# HEALTH. The base image is distroless — measured against this digest, not even
|
||||
# `ls` exists (`exec: "ls": executable file not found in $PATH`), so there is no
|
||||
# `/bin/sh`, no curl and no wget. SurrealDB ships its own probe instead:
|
||||
# `surreal is-ready` connects to ws://localhost:8000 by default, needs no
|
||||
# credentials, prints `OK` and exits 0 while the server is up.
|
||||
#
|
||||
# It must be handed over as a QUADLET-SIDE JSON ARRAY, not a bare command.
|
||||
# podman wraps a bare `HealthCmd` string as `CMD-SHELL` (measured on this host's
|
||||
# podman 5.7: `--health-cmd "/surreal is-ready"` stores
|
||||
# `{"Test":["CMD-SHELL","/surreal is-ready"]}`), and with no `/bin/sh` in the
|
||||
# image that probe fails in ~30 ms with empty output on every interval while the
|
||||
# database answers perfectly — which is exactly how this service's first deploy
|
||||
# died on 2026-10-19 (`did not become healthy in 90s`). The quoted array survives
|
||||
# the generator (`--health-cmd "[\"/surreal\",\"is-ready\"]"`) and stores
|
||||
# `["CMD","/surreal","is-ready"]`, which execs directly and goes healthy. Do not
|
||||
# "simplify" it back to a bare command.
|
||||
|
||||
[Unit]
|
||||
Description=SurrealDB (open-notebook datastore)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=open-notebook-surrealdb
|
||||
Image=docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||
Pull=never
|
||||
Network=open-notebook.network
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||
Exec=start --log info rocksdb:/mydata/mydatabase.db
|
||||
Volume=open-notebook-db:/mydata
|
||||
HealthCmd=["/surreal","is-ready"]
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=20s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,105 +0,0 @@
|
||||
# Quadlet unit: open-notebook (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.container
|
||||
# Unit name: open-notebook.service (container name: open-notebook)
|
||||
#
|
||||
# WHAT IT IS. The fleet's podcast generator: notes and sources go in, a
|
||||
# two-voice (up to four) episode MP3 comes out. One image runs THREE processes
|
||||
# under supervisord — the FastAPI API on 5055, the background worker that
|
||||
# writes and speaks the episode, and the Next.js UI on 8502 — so this one unit
|
||||
# owns the whole app, and the frontend refuses to start until the API answers
|
||||
# (the image's own wait-for-api.sh). Runbook: [[Open Notebook]].
|
||||
#
|
||||
# IMAGE. `lfnovo/open_notebook:v1-latest` pinned by manifest-list digest,
|
||||
# resolved from the registry on 2026-10-18; it is the tag upstream's own
|
||||
# docker-compose.yml pins for the multi-container shape. Do NOT reach for the
|
||||
# `-single` tags: that variant bundles SurrealDB inside the app image, is
|
||||
# deprecated upstream, and would give this host two databases. Pull=never, so
|
||||
# the pin is fetched by the documented one-time step (docs: [[Open Notebook]]),
|
||||
# which the deployer also runs when the image is absent:
|
||||
#
|
||||
# podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||
#
|
||||
# NETWORKS, in this order on purpose. The generated run line keeps file order,
|
||||
# so the DEFAULT ROUTE rides dankclaw_default: that is the network where this
|
||||
# host has measured that a bridge container can reach a host-namespace
|
||||
# listener through the gateway name (the Prometheus scrape of frank-zealot is
|
||||
# the proof — see [[Container Network Namespaces]]), and frank-tts lives there
|
||||
# too, so the TTS credential can be `http://frank-tts:7772/v1`. The private
|
||||
# systemd-open-notebook bridge is second and carries only the database; its
|
||||
# traffic is on-link, so losing the default route costs it nothing.
|
||||
#
|
||||
# PUBLISHED PORTS, and the 5055 collision. Upstream publishes 8502 and 5055,
|
||||
# and 5055 is exactly the port seerr already holds on this host
|
||||
# (`PublishPort=127.0.0.1:5055:5055`; `ss -ltn` on Dankclaw 2026-10-18 shows
|
||||
# 127.0.0.1:5055 taken). So the API leaves the container on host port 5056.
|
||||
# Only 8502 is published on all interfaces, because it is the single upstream
|
||||
# the guard chain dials: notebook.franksdank.cloud → anubis → authentik →
|
||||
# http://:8017 → 192.168.0.143:8502 (dankclaw/Caddyfile), and this Caddy lives
|
||||
# in root's storage, so a published port is the only way it can reach a rootless
|
||||
# quadlet. The API needs no outward face at all — the image's own Next.js layer
|
||||
# rewrites /api/* to the FastAPI beside it, so one origin serves UI and API —
|
||||
# and the fleet's rule is that loopback-only is for services whose user path is
|
||||
# the guard chain, which this became on 2026-10-19; 5056 stays published on
|
||||
# loopback because the deployer's health gate dials it there. That is also why
|
||||
# API_URL in scripts/open-notebook.env names the vhost rather than a port: left
|
||||
# to itself the frontend auto-detects `<host-you-opened>:5055`, and the UI loads
|
||||
# while every call fails. OPEN_NOTEBOOK_PASSWORD is not made redundant by the
|
||||
# chain: it is what keeps the API honest for everything else on this host, and
|
||||
# with it unset the API answers anyone, including the unauthenticated
|
||||
# episode-audio download.
|
||||
#
|
||||
# VOLUMES. `/app/data` is a named volume: uploads, the LangGraph SQLite
|
||||
# checkpoints and the extraction caches the image keeps under it
|
||||
# (UV_CACHE_DIR, HF_HOME) are app state nobody browses. The ONE exception is
|
||||
# finished audio, which is bind-mounted out of the container at
|
||||
# /home/dank/media/podcasts. That mount is interim and reversible: episode
|
||||
# destination is an open human-only decision (Gitea issue #7, label
|
||||
# `human-only` — agents must not work it), and until it lands, episodes are
|
||||
# ordinary files on the host. DATA_FOLDER is hard-coded `./data` in the app
|
||||
# (WORKDIR /app) and the DB stores audio paths RELATIVE to data/podcasts, so
|
||||
# covering that one subdirectory is enough and removing the line changes
|
||||
# nothing else. The image declares no USER, so the app runs as container root,
|
||||
# which in this host's rootless mapping (0 -> 1000, measured on this host —
|
||||
# see [[Seerr]]) writes those episode files as dank.
|
||||
#
|
||||
# HEALTH. `GET /health` on the API returns {"status":"healthy"} and is in the
|
||||
# password middleware's excluded-path list, so the probe stays honest after the
|
||||
# password is set (both read from api/main.py upstream). The image installs
|
||||
# curl and ffmpeg in the runtime layer, so the probe needs no shell — kept
|
||||
# argument-only for the generator's space-splitting. Note what this probe does
|
||||
# NOT prove: the route returns a static body, so it says the API serves, not
|
||||
# that the database answers. The DB has its own probe in its unit, and the
|
||||
# worker's failure to reach the database shows up as failed jobs, not as an
|
||||
# unhealthy container.
|
||||
|
||||
[Unit]
|
||||
Description=open-notebook (podcast generator: API, worker, web UI)
|
||||
After=network-online.target open-notebook-surrealdb.service
|
||||
Wants=network-online.target open-notebook-surrealdb.service
|
||||
|
||||
[Container]
|
||||
ContainerName=open-notebook
|
||||
Image=docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
Network=open-notebook.network
|
||||
PublishPort=0.0.0.0:8502:8502
|
||||
PublishPort=127.0.0.1:5056:5055
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||
Volume=open-notebook-data:/app/data
|
||||
Volume=/home/dank/media/podcasts:/app/data/podcasts
|
||||
HealthCmd=curl -fsS http://127.0.0.1:5055/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=60s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
TimeoutStartSec=900
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,34 +0,0 @@
|
||||
# Quadlet network: private bridge for the open-notebook pair (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.network
|
||||
# Unit name: open-notebook-network.service (podman network name:
|
||||
# systemd-open-notebook — quadlet prefixes "systemd-", so probe the runtime
|
||||
# name, never the filename; see [[Container Network Namespaces]]).
|
||||
#
|
||||
# WHY A PRIVATE BRIDGE. The app reaches its database by container name
|
||||
# (SURREAL_URL=ws://open-notebook-surrealdb:8000/rpc), and container DNS names
|
||||
# resolve only on the network that owns them. Putting SurrealDB here instead of
|
||||
# on dankclaw_default keeps the datastore reachable ONLY to the container that
|
||||
# owns it: 8000 is not published (an unpublished port is refused outright from
|
||||
# every other namespace anyway), and nothing else on the fleet-wide network can
|
||||
# dial it with the root credentials. Same reasoning as the gitea and authentik
|
||||
# database pairs.
|
||||
#
|
||||
# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
|
||||
# per-container userspace helper left a container that accepted TCP and reset
|
||||
# every request for two days, logging nothing). A bridge has no such helper.
|
||||
#
|
||||
# Subnet registry on dankclaw (rootless):
|
||||
# 10.88.0.0/16 default "podman" bridge
|
||||
# 10.89.0.0/24 gitea (systemd-gitea)
|
||||
# 10.89.1.0/24 compose stack (dankclaw_default)
|
||||
# 10.89.2.0/24 searxng (systemd-searxng)
|
||||
# 10.89.3.0/24 open-webui (systemd-open-webui)
|
||||
# 10.89.4.0/24 open-notebook (systemd-open-notebook) <- this file
|
||||
# Pinned so the container IPs stay deterministic across reboots, like the
|
||||
# other pinned nets; 10.89.4.0/24 was the next unclaimed /24 in this registry.
|
||||
|
||||
[Network]
|
||||
Driver=bridge
|
||||
Subnet=10.89.4.0/24
|
||||
Gateway=10.89.4.1
|
||||
@@ -1,96 +0,0 @@
|
||||
# Quadlet unit: open-webui (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/open-webui.container
|
||||
# Unit name: open-webui.service (container name: open-webui)
|
||||
#
|
||||
# Chat UI for the LLM fleet. It runs on the CPU image (dankclaw is CPU-only, so
|
||||
# the :cuda image would waste a large pull for a GPU it cannot use) and points
|
||||
# its OpenAI connection at the live llama.cpp backend on this host via the
|
||||
# host LAN IP (192.168.0.143:7770, model frank-default-large). It publishes 3005
|
||||
# (3000 is Gitea's HTTP port). Data persists under
|
||||
# /home/dank/Containers/open-webui/data. See serverAppConfigs-docs/Dankclaw/OpenWebUI.md.
|
||||
#
|
||||
# Network history (2026-08-22): this unit used Network=slirp4netns. When the
|
||||
# slirp4netns helper process died, the container kept running but lost its tap
|
||||
# interface: a zombie that accepted TCP and reset every request, and chat
|
||||
# served empty responses for ~2 days with no error anywhere. It now runs on a
|
||||
# pinned bridge network (open-webui.network), which has no per-container
|
||||
# helper to die. The LLM is reached via the host LAN IP (192.168.0.143):
|
||||
# the bridge gateway lives in the rootless netns, not the host netns where
|
||||
# llama-server runs. The healthcheck probes /health (app liveness) and
|
||||
# requires a non-loopback interface (network liveness).
|
||||
|
||||
[Unit]
|
||||
Description=open-webui chat UI container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=open-webui
|
||||
Network=open-webui.network
|
||||
Image=ghcr.io/open-webui/open-webui:main
|
||||
PublishPort=3005:8080
|
||||
Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
|
||||
Environment=ANONYMIZED_TELEMETRY=False
|
||||
Environment=DO_NOT_TRACK=True
|
||||
|
||||
# SSO via authentik (OIDC authorization-code flow, 2026-09-24). The OAuth
|
||||
# client (application slug "open-webui") lives in authentik; the client
|
||||
# SECRET stays off-repo in the git-ignored host env file (mode 600). The
|
||||
# container reaches the IdP over the public URL: Caddy publishes 0.0.0.0:443
|
||||
# and Pi-hole resolves auth.franksdank.cloud to the host LAN IP, so the
|
||||
# server-side code exchange works from this bridge. Roles come from the
|
||||
# "Open WebUI roles" scope mapping (entitlements "Open WebUI Users" /
|
||||
# "Open WebUI Admins"); OAUTH_MERGE_ACCOUNTS_BY_EMAIL folds SSO logins into
|
||||
# the existing local account (frank8465@protonmail.com). The OAuth vars are
|
||||
# read directly from the environment at runtime (the per-key config DB holds
|
||||
# only Admin-Panel overrides, so nothing is seeded on the live data dir).
|
||||
# NOTE: the OAUTH_SCOPES value MUST stay quoted — systemd splits unquoted
|
||||
# Environment= values on whitespace (unquoted, only "openid" survived).
|
||||
EnvironmentFile=/home/dank/Containers/open-webui/env
|
||||
Environment=OAUTH_CLIENT_ID=KmtzKc0bjWjaIFeM0puYudI1CLXR2p0jnPCSf4N0
|
||||
Environment=OAUTH_PROVIDER_NAME=authentik
|
||||
Environment=OPENID_PROVIDER_URL=https://auth.franksdank.cloud/application/o/open-webui/.well-known/openid-configuration
|
||||
Environment=OPENID_REDIRECT_URI=https://ai.franksdank.cloud/oauth/oidc/callback
|
||||
Environment=WEBUI_URL=https://ai.franksdank.cloud
|
||||
Environment=ENABLE_OAUTH_SIGNUP=true
|
||||
Environment=OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true
|
||||
Environment="OAUTH_SCOPES=openid email profile roles"
|
||||
Environment=ENABLE_OAUTH_ROLE_MANAGEMENT=true
|
||||
# SSO-only (2026-09-24): the OIDC round-trip is verified, so the local
|
||||
# username/password path is retired. ENABLE_LOGIN_FORM hides the form;
|
||||
# ENABLE_PASSWORD_AUTH refuses password logins entirely, so the only way in
|
||||
# is the authentik OIDC flow (which folds into the local account by email).
|
||||
# The per-key config DB override (ui.enable_login_form) is flipped to false
|
||||
# in the same change — the DB wins over env, so both must agree.
|
||||
Environment=ENABLE_LOGIN_FORM=false
|
||||
Environment=ENABLE_PASSWORD_AUTH=false
|
||||
|
||||
# Web search -> SearXNG, reached over the host-published port (NOT the
|
||||
# search.franksdank.cloud vhost): the Anubis->Authentik guard chain is a
|
||||
# browser mechanism and a plain HTTP client cannot solve it. The internal
|
||||
# port bypasses Caddy entirely. These env vars SEED the per-key config DB on
|
||||
# first run (fill-missing only); once the keys exist in /app/backend/data the
|
||||
# DB is authoritative, so this block documents the intent and makes a fresh
|
||||
# data dir come up wired. SearXNG's limiter exempts this source (the bridge
|
||||
# egress 10.89.x is in pass_ip), so the 4-req/IP/hour JSON cap never applies.
|
||||
Environment=ENABLE_WEB_SEARCH=True
|
||||
Environment=WEB_SEARCH_ENGINE=searxng
|
||||
Environment=SEARXNG_QUERY_URL=http://192.168.0.143:8081/search
|
||||
Environment=SEARXNG_LANGUAGE=all
|
||||
Environment=WEB_SEARCH_RESULT_COUNT=6
|
||||
Volume=/home/dank/Containers/open-webui/data:/app/backend/data
|
||||
|
||||
# App liveness (/health) AND network liveness (a non-loopback interface must
|
||||
# exist; a dead tap leaves only "lo" behind).
|
||||
HealthCmd=CMD-SHELL python -c 'import os,urllib.request,sys; urllib.request.urlopen("http://127.0.0.1:8080/health",timeout=5).read(); sys.exit(0 if any(i!="lo" for i in os.listdir("/sys/class/net")) else 1)'
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=300s
|
||||
|
||||
[Service]
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,31 +0,0 @@
|
||||
# Quadlet network: private bridge for open-webui (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/open-webui.network
|
||||
# Unit name: open-webui-network.service (podman network name: systemd-open-webui)
|
||||
#
|
||||
# Why a bridge instead of slirp4netns (2026-08-22 incident):
|
||||
# slirp4netns runs a user-space helper process per container. When that
|
||||
# helper died, the container kept running but lost its tap interface,
|
||||
# becoming a zombie that accepted TCP and reset every request. chat
|
||||
# served empty responses for ~2 days and nothing logged an error.
|
||||
# A kernel bridge has no per-container helper, so the failure mode
|
||||
# cannot occur.
|
||||
#
|
||||
# The subnet is pinned so the container's own IP (10.89.3.2) is
|
||||
# deterministic. Note the LLM backend is NOT reached via this gateway:
|
||||
# the gateway lives in the rootless netns, while llama-server runs in
|
||||
# the host netns. The host's LAN IP (192.168.0.143) reaches it through
|
||||
# the rootless NAT. host.containers.internal is a slirp4netns-only name
|
||||
# and does not resolve on a bridge.
|
||||
#
|
||||
# Subnet registry on dankclaw (rootless):
|
||||
# 10.88.0.0/16 default "podman" bridge
|
||||
# 10.89.0.0/24 gitea (systemd-gitea)
|
||||
# 10.89.1.0/24 compose stack (dankclaw_default)
|
||||
# 10.89.2.0/24 searxng (systemd-searxng)
|
||||
# 10.89.3.0/24 open-webui (systemd-open-webui) <- this file
|
||||
|
||||
[Network]
|
||||
Driver=bridge
|
||||
Subnet=10.89.3.0/24
|
||||
Gateway=10.89.3.1
|
||||
@@ -1,48 +0,0 @@
|
||||
# A records for the flat franksdank.name scheme (dnsmasq addn-hosts format:
|
||||
# <IP> <name> [<name> ...]). Served by the Pi-hole quadlet (FTL is a
|
||||
# dnsmasq fork): the quadlet mounts this file into /etc/pihole/hosts/ read-only.
|
||||
#
|
||||
# Caddy is consolidated on dankclaw (192.168.0.143): it fronts every service
|
||||
# on both hosts, so every name resolves to that one IP. Keep in sync with the
|
||||
# Caddyfile (tests/pihole.bats cross-checks this).
|
||||
#
|
||||
# 192.168.0.143 = dankclaw (runs the Caddy container + the AI/dev stack)
|
||||
# 192.168.0.235 = dankserver (media stack; reached by Caddy via published
|
||||
# LAN ports — no DNS records point here)
|
||||
|
||||
# --- Homepage (naked domain + www) + legacy names ---
|
||||
192.168.0.143 franksdank.cloud
|
||||
192.168.0.143 www.franksdank.cloud
|
||||
192.168.0.143 dankclaw.franksdank.cloud
|
||||
192.168.0.143 dankserver.franksdank.cloud
|
||||
|
||||
# --- Authentik (SSO / OIDC provider) ---
|
||||
192.168.0.143 auth.franksdank.cloud
|
||||
|
||||
# --- Media services (dankserver, proxied) ---
|
||||
192.168.0.143 plex.franksdank.cloud
|
||||
192.168.0.143 plex.franksdank.casa
|
||||
192.168.0.143 books.franksdank.cloud
|
||||
192.168.0.143 romm.franksdank.cloud
|
||||
192.168.0.143 files.franksdank.cloud
|
||||
192.168.0.143 jelly.franksdank.cloud
|
||||
192.168.0.143 radarr.franksdank.cloud
|
||||
192.168.0.143 lidarr.franksdank.cloud
|
||||
192.168.0.143 readarr.franksdank.cloud
|
||||
192.168.0.143 prowlarr.franksdank.cloud
|
||||
192.168.0.143 qbittorrent.franksdank.cloud
|
||||
|
||||
# --- AI / search / dev services (dankclaw, local) ---
|
||||
192.168.0.143 search.franksdank.cloud
|
||||
192.168.0.143 comfyui.franksdank.cloud
|
||||
192.168.0.143 ai.franksdank.cloud
|
||||
192.168.0.143 hermes.franksdank.cloud
|
||||
192.168.0.143 hermesadmin.franksdank.cloud
|
||||
192.168.0.143 gitea.franksdank.cloud
|
||||
192.168.0.143 grafana.franksdank.cloud
|
||||
192.168.0.143 prometheus.franksdank.cloud
|
||||
192.168.0.143 loki.franksdank.cloud
|
||||
192.168.0.143 pihole.franksdank.cloud
|
||||
192.168.0.143 seerr.franksdank.cloud
|
||||
192.168.0.143 marreta.franksdank.cloud
|
||||
192.168.0.143 notebook.franksdank.cloud
|
||||
@@ -1,18 +0,0 @@
|
||||
# Pi-hole container environment (dankclaw).
|
||||
# Copy to /home/dank/Containers/pihole/env and fill real values.
|
||||
# The real env file is never committed (it holds the web UI password).
|
||||
|
||||
# The host IP the container answers for (also shown in the web UI).
|
||||
ServerIP=192.168.0.143
|
||||
|
||||
# Timezone for the web UI and logs.
|
||||
TZ=America/Los_Angeles
|
||||
|
||||
# Pinned upstream resolvers. The slirp netns has no useful resolv.conf, so
|
||||
# pin them (mirrors the retired dnsmasq.conf server= lines).
|
||||
DNS1=1.1.1.1
|
||||
DNS2=1.0.0.1
|
||||
|
||||
# Web UI password (the shared family password; the same credential as
|
||||
# CADDY_BASICAUTH_HASH in the host .env).
|
||||
WEBPASSWORD=changeme
|
||||
@@ -1,42 +0,0 @@
|
||||
# Quadlet unit: Pi-hole LAN DNS (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/pihole.container
|
||||
# Unit name: pihole.service (container name: pihole)
|
||||
#
|
||||
# The LAN's authoritative DNS for the flat franksdank.cloud / franksdank.casa
|
||||
# name scheme. It retired the dnsmasq quadlet (2026-08-24): Pi-hole FTL is a
|
||||
# dnsmasq fork, so the addn-hosts record file carries over unchanged and is
|
||||
# mounted into /etc/pihole/hosts/ (the FTL hostsdir) read-only. It serves one A
|
||||
# record per Caddy vhost and forwards everything else to the pinned upstream
|
||||
# resolvers (DNS1/DNS2 in the env file). The SURFboard G36 router hands this
|
||||
# server out to DHCP clients (DNS statically configured -> 192.168.0.143),
|
||||
# so no client needs per-device config.
|
||||
#
|
||||
# Runs on slirp4netns with port 53 (DNS) and 8083 (web UI) published to the
|
||||
# LAN (the same pattern as open-webui). Config + state mount from
|
||||
# /home/dank/Containers/pihole. The web UI password is WEBPASSWORD in the env
|
||||
# file. See serverAppConfigs-docs/Infrastructure/LAN-DNS.md.
|
||||
|
||||
[Unit]
|
||||
Description=Pi-hole LAN DNS container
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=pihole
|
||||
Image=docker.io/pihole/pihole:latest
|
||||
Network=slirp4netns
|
||||
PublishPort=0.0.0.0:53:53/udp
|
||||
PublishPort=0.0.0.0:53:53/tcp
|
||||
PublishPort=0.0.0.0:8083:80/tcp
|
||||
EnvironmentFile=/home/dank/Containers/pihole/env
|
||||
Volume=/home/dank/Containers/pihole/etc-pihole:/etc/pihole
|
||||
Volume=/home/dank/Containers/pihole/etc-dockers:/etc/dockers
|
||||
Volume=/home/dank/Containers/pihole/addn-hosts:/etc/pihole/hosts/addn-hosts:ro
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,8 +0,0 @@
|
||||
# Rootless DNS must bind port 53: lower the unprivileged-port threshold so a
|
||||
# rootless container may publish 0.0.0.0:53 (80/443 stay allowed; 53 < 80).
|
||||
# Single source of truth for net.ipv4.ip_unprivileged_port_start on dankclaw;
|
||||
# deploy_pihole installs it to /etc/sysctl.d/ and applies it with
|
||||
# `sysctl --system` (the same code path boot uses via systemd-sysctl).
|
||||
# A later-sorted drop-in that reset this to 80 took LAN DNS down after the
|
||||
# 2026-08-26 reboot — deploy_pihole neutralizes such conflicts.
|
||||
net.ipv4.ip_unprivileged_port_start=53
|
||||
@@ -1,40 +0,0 @@
|
||||
# Quadlet unit: Gluetun VPN tunnel (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/gluetun.container
|
||||
# Unit name: gluetun.service (container name: gluetun)
|
||||
#
|
||||
# Creates the WireGuard tunnel to ProtonVPN (US-WA). SearXNG reuses this
|
||||
# container's network namespace (see searxng.container), so its outbound search
|
||||
# traffic exits here. The WireGuard credentials live in /opt/searxng/.env
|
||||
# (mode 0600); this file holds no secrets. See serverAppConfigs-docs/Dankclaw/Gluetun.md.
|
||||
|
||||
[Unit]
|
||||
Description=Gluetun VPN tunnel container
|
||||
Wants=searxng-network.service
|
||||
After=searxng-network.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=gluetun
|
||||
Network=searxng.network
|
||||
Image=docker.io/qmcgaw/gluetun:latest
|
||||
AddCapability=NET_ADMIN
|
||||
AddDevice=/dev/net/tun:/dev/net/tun
|
||||
Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
|
||||
Environment=VPN_TYPE=${VPN_TYPE}
|
||||
Environment=WIREGUARD_PRIVATE_KEY=${WIREGUARD_PRIVATE_KEY}
|
||||
Environment=WIREGUARD_ADDRESSES=${WIREGUARD_ADDRESSES}
|
||||
Environment=WIREGUARD_PUBLIC_KEY=${WIREGUARD_PUBLIC_KEY}
|
||||
Environment=WIREGUARD_ENDPOINT_IP=${WIREGUARD_ENDPOINT_IP}
|
||||
Environment=WIREGUARD_ENDPOINT_PORT=${WIREGUARD_ENDPOINT_PORT}
|
||||
Environment=WIREGUARD_ALLOWED_IPS=${WIREGUARD_ALLOWED_IPS}
|
||||
Environment=WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=${WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL}
|
||||
PublishPort=8081:8080
|
||||
PublishPort=8082:8085
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/opt/searxng/.env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,28 +0,0 @@
|
||||
# Quadlet unit: Valkey cache for SearXNG (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/searxng-valkey.container
|
||||
# Unit name: searxng-valkey.service (container name: searxng-valkey)
|
||||
#
|
||||
# The limiter's rate-limit state store. Joins the searxng network and is
|
||||
# reachable by the name "valkey" (settings.yml points there). Data persists in
|
||||
# /opt/searxng/valkey. See serverAppConfigs-docs/Dankclaw/Valkey.md.
|
||||
|
||||
[Unit]
|
||||
Description=SearXNG Valkey cache container
|
||||
Wants=searxng-network.service
|
||||
After=searxng-network.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=searxng-valkey
|
||||
Network=searxng.network
|
||||
NetworkAlias=valkey
|
||||
Image=docker.io/valkey/valkey:9-alpine
|
||||
Volume=/opt/searxng/valkey:/data
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/opt/searxng/.env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,30 +0,0 @@
|
||||
# Quadlet unit: SearXNG search engine (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/searxng.container
|
||||
# Unit name: searxng.service (container name: searxng)
|
||||
#
|
||||
# Reuses gluetun's network namespace (Network=gluetun.container) so its
|
||||
# outbound traffic exits the WireGuard tunnel. It publishes no port of its own;
|
||||
# the port lives on gluetun. It reaches valkey by name on the searxng network.
|
||||
# Config mounts from /opt/searxng/config. See serverAppConfigs-docs/Dankclaw/SearXNG.md.
|
||||
|
||||
[Unit]
|
||||
Description=SearXNG search engine container
|
||||
Wants=gluetun.service searxng-valkey.service
|
||||
After=gluetun.service searxng-valkey.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=searxng
|
||||
Network=gluetun.container
|
||||
Image=ghcr.io/searxng/searxng:latest
|
||||
Environment=SEARXNG_SECRET=${SEARXNG_SECRET}
|
||||
Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL}
|
||||
Volume=/opt/searxng/config:/etc/searxng
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/opt/searxng/.env
|
||||
TimeoutStartSec=900
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,16 +0,0 @@
|
||||
# Quadlet network: private bridge for the SearXNG stack (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/searxng.network
|
||||
# Unit name: searxng-network.service (podman network name: systemd-searxng)
|
||||
#
|
||||
# DNS is enabled by default for quadlet networks. This matters: the automatic
|
||||
# "podman" default network has dns_enabled=false, so container names would not
|
||||
# resolve. searxng (which reuses gluetun's netns) reaches valkey by the name
|
||||
# "valkey" on this network.
|
||||
#
|
||||
# The subnet is pinned so the value stays stable across reboots. It must match
|
||||
# an entry in shared/searxng/limiter.toml trusted_proxies. 10.89.2.0/24 is free
|
||||
# (gitea uses 10.89.0.0/24; the old compose net used 10.89.1.0/24).
|
||||
|
||||
[Network]
|
||||
Subnet=10.89.2.0/24
|
||||
@@ -1,43 +0,0 @@
|
||||
# Quadlet unit: Seerr media request manager (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/seerr.container
|
||||
# Unit name: seerr.service (container name: seerr)
|
||||
#
|
||||
# Seerr is the unified Overseerr/Jellyseerr successor. It fronts the
|
||||
# *arr stack now living on FrankBook-Pro (192.168.0.112, resolved via the
|
||||
# Pi-hole addn-hosts record frankbook-pro.lan): requests are placed here,
|
||||
# routed through the arrs over the LAN, and downloads run inside the Mac's
|
||||
# gluetun tunnel. SQLite lives under /home/dank/Containers/seerr/data —
|
||||
# no separate database unit (decision D8).
|
||||
#
|
||||
# Exposure contract: the ONLY published port is loopback (127.0.0.1:5055)
|
||||
# for the homepage probe and host-local checks. User traffic must traverse
|
||||
# the guard chain: Caddy TLS edge → Anubis :8923 → Authentik outpost →
|
||||
# Caddy :8017 post-auth listener → seerr:5055 on dankclaw_default. Never
|
||||
# publish 5055 on a LAN or public address.
|
||||
#
|
||||
# Image pin: ghcr.io/seerr-team/seerr:v3.4.1 (manifest-verified on GHCR
|
||||
# 2026-09-01; released 2026-07-30). Upgrade = bump the tag, redeploy.
|
||||
#
|
||||
# No secrets in this file: OIDC (Authentik) and arr API keys live in
|
||||
# Seerr's own settings store inside the data volume, configured at runtime.
|
||||
|
||||
[Unit]
|
||||
Description=Seerr media request manager container
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=seerr
|
||||
Image=ghcr.io/seerr-team/seerr:v3.4.1
|
||||
Network=dankclaw_default
|
||||
PublishPort=127.0.0.1:5055:5055
|
||||
Volume=/home/dank/Containers/seerr/data:/app/config
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStartSec=900
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,26 +0,0 @@
|
||||
# /etc/udev/rules.d/98-halogen-npu-access.rules
|
||||
#
|
||||
# Why the NPU node is world-writable: `frank-dragoon` runs rootless podman under a
|
||||
# unit with `User=dank`, and the container's user namespace maps exactly one uid and
|
||||
# one gid (dank). `/dev/accel/accel0` is `root:render 0660`, so inside the container
|
||||
# the node reads `nobody:nogroup` and the engine dies with its own diagnosis:
|
||||
#
|
||||
# halogen npu: /dev/accel/accel0 is here, but this process cannot open it
|
||||
# (crw-rw---- nobody:nogroup).
|
||||
#
|
||||
# `--group-add keep-groups` is the vendor's advice and it does NOT work in this
|
||||
# shape: measured 2026-10-19 from the host as a `dank` shell that was already in
|
||||
# `render`, with and without the flag, the node still read `nobody:nogroup` and the
|
||||
# open was refused — an unmapped gid cannot be granted to a process inside a
|
||||
# namespace that does not map it. Same conclusion `99-comfyui-gpu-access.rules`
|
||||
# reached for `/dev/kfd` from the systemd-user-spawn direction: the mode is the only
|
||||
# spawn-context-independent door short of running the stack rootful.
|
||||
#
|
||||
# Blast radius: any process in this host's single uid domain (every rootless
|
||||
# container here shares it) can submit to the XDNA2 NPU. The NPU holds no user data
|
||||
# and no privileged context — the same trade already accepted for the GPU compute
|
||||
# nodes beside it. Installed by `deploy-remote.sh` when the NPU lane is armed and left
|
||||
# in place afterwards — the mode costs nothing while the lane is idle. `serve-frank-dragoon.sh`
|
||||
# refuses to arm the lane on a node that is not other-accessible, so a host that lost this
|
||||
# file fails loudly at boot instead of crash-looping.
|
||||
KERNEL=="accel[0-9]*", SUBSYSTEM=="accel", MODE="0666"
|
||||
@@ -1,21 +0,0 @@
|
||||
# /etc/udev/rules.d/99-comfyui-gpu-access.rules
|
||||
#
|
||||
# Why world-writable compute nodes: the comfyui container is spawned by a
|
||||
# systemd user unit (comfyui.service, Type=oneshot). User-unit processes are
|
||||
# spawned by the user manager with NO supplementary groups, so any spawn path
|
||||
# through systemd loses `render`/`video` membership regardless of how the
|
||||
# container is launched (keep-groups and named group_add both depend on the
|
||||
# spawning process already holding the group — verified 2026-08-31: keep-groups
|
||||
# preserved an empty group list under the user manager and /dev/kfd open still
|
||||
# failed with EACCES). A mode that grants access to "everyone in this uid
|
||||
# domain" is the only spawn-context-independent option short of running the
|
||||
# stack rootful.
|
||||
#
|
||||
# Blast radius: any local process (every container here runs as the same uid
|
||||
# domain) can already ptrace the GPU userspace through an existing holder's
|
||||
# fd inheritance; on a private single-administrator home server this trades a
|
||||
# capability gate that could never be held by the intended user anyway.
|
||||
# Note the missing SUBSYSTEM= guard on the kfd line: misc-class devices have no
|
||||
# subsystem for udev matching, so a SUBSYSTEM=="misc" guard never matches.
|
||||
KERNEL=="kfd", MODE="0666"
|
||||
KERNEL=="renderD*", SUBSYSTEM=="drm", MODE="0666"
|
||||
@@ -1,43 +0,0 @@
|
||||
# frank-tts — the cloned-voice speech façade (dankclaw/voiceservice/app.py).
|
||||
#
|
||||
# Standard library only, so there is nothing to pip-install and nothing to
|
||||
# compile: the image is a python runtime plus ffmpeg (the one binary the app
|
||||
# shells out to, for response_format=mp3) plus one COPY of app.py. Rebuild to
|
||||
# ship a new app.py.
|
||||
#
|
||||
# Build on dankclaw (deploy-remote.sh build_compose_image frank-tts does this,
|
||||
# including --format docker, which the fleet's other local builds also pass):
|
||||
# podman build --format docker -t franksdank/frank-tts:latest \
|
||||
# -f dankclaw/voiceservice/Containerfile .
|
||||
#
|
||||
# BASE PINNED BY DIGEST, like every other image this fleet runs from a
|
||||
# registry: a tag moves under a rebuild and a voice endpoint that changes
|
||||
# python or ffmpeg mid-week is a debug session nobody asked for. The digest is
|
||||
# the multi-arch INDEX of python:3.12-alpine as published 2026-10-18 (amd64 +
|
||||
# arm64 on linux); Dankclaw resolves the amd64 entry.
|
||||
#
|
||||
# WHY 3.12: the fleet's ceiling for this app is python <= 3.12 (single file,
|
||||
# no build step); the alpine flavour also keeps the image in the tens of MB
|
||||
# and ships busybox `wget`, which is what the quadlet unit's HealthCmd runs —
|
||||
# so no HEALTHCHECK belongs here (the probe lives in the unit, as in
|
||||
# alertmanager/prometheus; only distroless bases need it baked in).
|
||||
FROM docker.io/library/python:3.12-alpine@sha256:1b668429b3511ab407d8e00648891631b0b1a4d7e15e3ca70f38ab5b91ad4ab4
|
||||
|
||||
# ffmpeg: mp3 output. The app always asks the upstream for wav and re-encodes
|
||||
# on the way out, so ffmpeg is the only external program in the picture.
|
||||
RUN apk add --no-cache ffmpeg
|
||||
|
||||
# Unbuffered stdout: the unit's journal must show the "listening on" line (and
|
||||
# any backend error) the moment it happens, not when the buffer fills.
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
COPY dankclaw/voiceservice/app.py /app/app.py
|
||||
|
||||
# Non-root. The voice packs are bind-mounted read-only, so this user only ever
|
||||
# reads them — they must be world-readable (the pack builder writes 644/755).
|
||||
# The one thing it writes is its per-request temp dir under /tmp.
|
||||
RUN adduser -D -u 10000 -h /app -s /sbin/nologin shim
|
||||
USER shim
|
||||
|
||||
EXPOSE 7772
|
||||
CMD ["python3", "/app/app.py"]
|
||||
@@ -1,178 +0,0 @@
|
||||
# frank-tts-engine — the arm A TTS engine: qwentts.cpp's resident `tts-server`
|
||||
# (the CPU binary that frank-tts on :7772 forwards to). The runbook is
|
||||
# [[Frank TTS Engine]]; the fleet-facing endpoint is [[Frank TTS]].
|
||||
#
|
||||
# WHAT THIS IS. The upstream source is NOT vendored here: the image clones
|
||||
# ServeurpersoCom/qwentts.cpp at one pinned commit and builds it. Rebuilding is
|
||||
# therefore a network operation (github.com plus objects.githubusercontent.com
|
||||
# for the `ggml` submodule), and build_compose_image in scripts/deploy-remote.sh
|
||||
# skips it whenever this Containerfile's sha256 label already matches the built
|
||||
# image — the same offline-safe idiom as loki/alloy/comfyui, so a re-deploy of
|
||||
# an unchanged engine needs no network.
|
||||
#
|
||||
# THE PINNED COMMIT IS THE CONTRACT. 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
|
||||
# is the commit the 2026-10-18 CPU trial on Dankclaw measured (server-side TTFA
|
||||
# 94-112 ms, warm RTF 0.55-0.67, run spread +-0.01 — the numbers that won the
|
||||
# engine decision). Moving it means re-measuring, not just editing this line.
|
||||
# `ggml` is a submodule of the repo (the only one; vendor/cpp-httplib and
|
||||
# vendor/yyjson are ordinary directories in the tree), and the submodule commit
|
||||
# is recorded in the parent tree, so cloning the parent commit recursively is
|
||||
# reproducible without a second pin.
|
||||
#
|
||||
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim — with the Vulkan door
|
||||
# open since 2026-10-19. That configure was run in this exact base image and
|
||||
# answers `Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version
|
||||
# "1.3.275") found components: glslc` then `Including Vulkan backend`:
|
||||
#
|
||||
# cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
|
||||
# -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
|
||||
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON \
|
||||
# -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
|
||||
# && cmake --build build -j 8
|
||||
#
|
||||
# Every flag is load-bearing, so do not "tidy" it:
|
||||
# GGML_BLAS=OFF is FORCED, not a preference — Dankclaw has libopenblas.so.0
|
||||
# but no BLAS dev headers, so the upstream default (BLAS on) fails to
|
||||
# configure. Upstream's own Dockerfile and buildcpu.sh both pass BLAS on,
|
||||
# which is why this file does not copy them.
|
||||
# GGML_CCACHE=OFF because a container build layer has no persistent ccache
|
||||
# directory to hit; on, it only writes a useless one.
|
||||
# GGML_OPENMP=ON is what makes the runtime need libgomp1 (installed below).
|
||||
# GGML_BACKEND_DL=OFF / GGML_CPU_ALL_VARIANTS=OFF: upstream's images build
|
||||
# every x86 ISA as a loadable backend so one image runs anywhere. This one
|
||||
# is built on the machine that runs it, so the single native CPU backend is
|
||||
# what was measured. That is also why no -DGGML_NATIVE is passed: ggml
|
||||
# defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
|
||||
# host — correct while build host == run host, wrong if anyone ever builds
|
||||
# this image elsewhere, which is why the fleet builds it on Dankclaw.
|
||||
# CUDA/HIP/METAL/SYCL/RPC stay OFF, VULKAN is ON: the endpoint computes on the
|
||||
# gfx1151 iGPU (decision and co-tenancy gates:
|
||||
# report/tts-npu-spike-verdict-2026-10-19.md, shape in [[Frank TTS Engine]]).
|
||||
# Nothing else is available to this image: no ROCm userland, no NVIDIA
|
||||
# userland, no macOS, no oneAPI, and RPC would add a second process to
|
||||
# supervise on a box that already has one engine per card.
|
||||
#
|
||||
# BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
|
||||
# tts-server links libggml*.so rather than swallowing them; the runtime stage
|
||||
# copies the binary and its shared objects out of the build tree and points
|
||||
# LD_LIBRARY_PATH at them, exactly as upstream's own Dockerfile does (the RPATH
|
||||
# baked at link time names the build directory, which does not exist here).
|
||||
#
|
||||
# BASE PINNED BY DIGEST, like every other image this fleet builds: the digest
|
||||
# is the multi-arch INDEX of ubuntu:24.04 as resolved from Docker Hub on
|
||||
# 2026-10-18; Dankclaw takes the amd64 entry. One base for both stages so there
|
||||
# is exactly one thing to re-resolve.
|
||||
#
|
||||
# NO HEALTHCHECK IN THIS FILE. The probe lives in the Quadlet unit
|
||||
# (HealthCmd=), which is the fleet pattern for anything with a shell — see the
|
||||
# alertmanager/prometheus units, and the opposite case in dankclaw/loki, whose
|
||||
# distroless image has no /bin/sh for the generator's CMD-SHELL wrapping.
|
||||
# `curl` is therefore installed in the runtime stage, for that probe only.
|
||||
|
||||
ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac
|
||||
|
||||
# ------------------------------------------------------------------ build
|
||||
FROM ${BASE_IMAGE} AS build
|
||||
|
||||
# The three Vulkan build deps are not optional and not interchangeable:
|
||||
# libvulkan-dev vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake
|
||||
# configs, i.e. what find_package(Vulkan) resolves to.
|
||||
# glslc ggml compiles its shaders with glslc, not glslangValidator.
|
||||
# Ubuntu's glslang-tools ships glslangValidator and NOT glslc,
|
||||
# so without this package `find_package(Vulkan COMPONENTS
|
||||
# glslc REQUIRED)` fails and there is no Vulkan backend.
|
||||
# The archive's glslc is old (2023.8): it compiles
|
||||
# GL_KHR_cooperative_matrix — the path ggml's quantized
|
||||
# matmul needs — but reports cooperative_matrix2,
|
||||
# integer_dot_product, bfloat16 and the micro-float
|
||||
# extensions as unsupported, so those shader families are
|
||||
# compiled out. If a Vulkan prefill ever reads as
|
||||
# inexplicably slow, suspect this compiler before the device;
|
||||
# the fix is a pinned glslang build passed as
|
||||
# -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
|
||||
# spirv-headers find_package(SPIRV-Headers CONFIG REQUIRED); the distro
|
||||
# package ships /usr/share/cmake/SPIRV-Headers/.
|
||||
RUN apt-get update -qq \
|
||||
&& apt-get install -y -qq --no-install-recommends \
|
||||
ca-certificates cmake g++ git make \
|
||||
libvulkan-dev glslc spirv-headers \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Checked out detached at a full SHA, then the recorded submodule. No branch
|
||||
# name anywhere: a branch moves, this commit does not.
|
||||
ARG QWENTTS_COMMIT=5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
|
||||
RUN git clone --no-checkout https://github.com/ServeurpersoCom/qwentts.cpp /src \
|
||||
&& git -C /src checkout --detach "${QWENTTS_COMMIT}" \
|
||||
&& git -C /src submodule update --init --depth 1 \
|
||||
&& git -C /src rev-parse HEAD \
|
||||
&& git -C /src submodule status
|
||||
|
||||
WORKDIR /src
|
||||
RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
|
||||
-DGGML_CCACHE=OFF \
|
||||
-DGGML_BACKEND_DL=OFF \
|
||||
-DGGML_CPU_ALL_VARIANTS=OFF \
|
||||
-DGGML_OPENMP=ON \
|
||||
-DGGML_BLAS=OFF \
|
||||
-DGGML_CUDA=OFF \
|
||||
-DGGML_HIP=OFF \
|
||||
-DGGML_VULKAN=ON \
|
||||
-DGGML_METAL=OFF \
|
||||
-DGGML_SYCL=OFF \
|
||||
-DGGML_RPC=OFF \
|
||||
&& cmake --build build -j 8
|
||||
|
||||
# ---------------------------------------------------------------- runtime
|
||||
FROM ${BASE_IMAGE}
|
||||
|
||||
# libgomp1: the OpenMP runtime the build above links against. curl: the unit's
|
||||
# HealthCmd. ca-certificates: so that curl can speak https if anyone ever
|
||||
# points a probe at a URL with a scheme this host terminates elsewhere.
|
||||
#
|
||||
# libvulkan1 + mesa-vulkan-drivers are what make `--device Vulkan0` mean
|
||||
# anything at runtime: libvulkan1 is the loader libggml-vulkan.so dlopens, and
|
||||
# mesa-vulkan-drivers is the RADV ICD that claims the gfx1151. Loader without an
|
||||
# ICD enumerates ZERO devices, and tts-server then dies in device lookup AFTER
|
||||
# both GGUFs have been read — the journal reads as a model fault, not a missing
|
||||
# driver. Enumerated from this base on 2026-10-19 with `--device /dev/dri
|
||||
# --group-add keep-groups` plus the host's ComfyUI udev rule: "Radeon 8060S
|
||||
# Graphics (RADV GFX1151)", API 1.4.318 (the heap figure is owned by
|
||||
# [[Frank TTS Engine]]; it is the GTT-backed aperture, not the ~2 GiB DRM
|
||||
# carve-out). No NVIDIA ICD is installed on purpose: one Vulkan device in the
|
||||
# enumeration is what makes the NAME Vulkan0 safe to write down.
|
||||
RUN apt-get update -qq \
|
||||
&& apt-get install -y -qq --no-install-recommends \
|
||||
ca-certificates curl libgomp1 libvulkan1 mesa-vulkan-drivers \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
|
||||
# model directory are host state mounted read-only, and one uid across the pair
|
||||
# means one thing to remember about what is world-readable.
|
||||
#
|
||||
# `useradd`, NOT `adduser`: this trimmed ubuntu:24.04 base ships no `adduser` at
|
||||
# all (verified 2026-10-18 against this digest — `adduser` is absent from the
|
||||
# base and is not pulled in by ca-certificates/curl/libgomp1 either), so an
|
||||
# `adduser` line dies at image-build time with crun's "executable file `adduser`
|
||||
# not found in $PATH". `useradd --system --uid 10000` succeeds and prints one
|
||||
# cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`); the uid is the
|
||||
# point, and it is what matches the façade. The alpine façade Containerfile keeps
|
||||
# its own busybox `adduser -D -u 10000` — different base, different tool, same uid.
|
||||
RUN useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=build /src/build/tts-server /src/build/*.so* /app/
|
||||
COPY dankclaw/voiceservice/tts-engine-start.sh /app/start.sh
|
||||
# The boot warmer lives INSIDE the image because the engine publishes no port:
|
||||
# the unit reaches it with `podman exec`, and the script needs the same curl,
|
||||
# base64 and /voices the engine's own namespace has. It is a sibling of
|
||||
# start.sh, not a second entrypoint — CMD stays the server.
|
||||
COPY dankclaw/voiceservice/tts-engine-warm.sh /app/warm-voices.sh
|
||||
RUN chmod 755 /app/start.sh /app/warm-voices.sh
|
||||
|
||||
# The binary's RPATH points inside the build tree; this is what makes it find
|
||||
# the libggml*.so copied beside it.
|
||||
ENV LD_LIBRARY_PATH=/app
|
||||
|
||||
USER tts
|
||||
EXPOSE 8880
|
||||
CMD ["/app/start.sh"]
|
||||
@@ -1,424 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""frank-tts — the fleet's OpenAI-compatible speech façade (component: the shim).
|
||||
|
||||
One local endpoint that answers ``voice=<name>`` with audio in a cloned voice.
|
||||
Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
|
||||
``POST /v1/audio/speech`` shape and never learn a file path: a voice is a
|
||||
directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
|
||||
the name real. A name that has no reference is not a voice — the shim answers
|
||||
400 with the list that is, instead of silently narrating in a fallback voice
|
||||
(silent fallback is how a wrong-voice endpoint stays wrong for weeks). A request
|
||||
that names NO voice at all takes ``$DEFAULT_VOICE``: that is a spoken choice in
|
||||
the env file rather than a surprise in the code, and the journal marks it
|
||||
``(default)`` so a name nobody asked for is never mistaken for one that was.
|
||||
|
||||
Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
|
||||
|
||||
``omni-http``
|
||||
Forward to an OpenAI-shaped upstream TTS server (``$TTS_UPSTREAM``) and
|
||||
carry the cloned voice with the request as ``reference_audio`` (base64 of
|
||||
``reference.wav``). The upstream does the inference; this process only
|
||||
resolves the voice and, when asked, re-encodes to mp3 with ``ffmpeg``.
|
||||
|
||||
``llama-tts-cli``
|
||||
Run a local binary per request (``$LLAMA_TTS_BIN``), hand it the
|
||||
reference with ``--tts-speaker-file``, read the WAV it writes. ``nice -n
|
||||
19`` because this box also serves text and a voice request must never
|
||||
starve it.
|
||||
|
||||
Everything comes from the environment, read at REQUEST time (no
|
||||
import-time-only globals): the same module can be imported with an injected
|
||||
``Config`` or run with ``python3 app.py`` and the unit's env file.
|
||||
|
||||
Runtime law: single process, standard library only, no build step. Audio
|
||||
bytes never reach the log — request lines and error text only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
WAV_CONTENT_TYPE = "audio/wav"
|
||||
MP3_CONTENT_TYPE = "audio/mpeg"
|
||||
|
||||
# A voice name is a plain directory name. Slashes and `..` are refused here so
|
||||
# a client-supplied name can never escape $VOICES_DIR.
|
||||
VOICE_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
|
||||
|
||||
|
||||
class ShimError(Exception):
|
||||
"""A failure with the HTTP status the client should see."""
|
||||
|
||||
def __init__(self, status, message):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
self.message = message
|
||||
|
||||
|
||||
class Config:
|
||||
"""Every knob of the shim. Constructed from the environment per request by
|
||||
default, or injected into ``create_server`` for tests and embedding."""
|
||||
|
||||
def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
|
||||
llama_model, llama_mmproj, llama_lang, llama_threads, port,
|
||||
max_chars, timeout, default_voice=""):
|
||||
self.voices_dir = Path(voices_dir)
|
||||
self.backend = backend
|
||||
self.upstream = upstream
|
||||
self.upstream_model = upstream_model
|
||||
self.llama_bin = llama_bin
|
||||
self.llama_model = llama_model
|
||||
self.llama_mmproj = llama_mmproj
|
||||
self.llama_lang = llama_lang
|
||||
self.llama_threads = llama_threads
|
||||
self.port = port
|
||||
self.max_chars = max_chars
|
||||
self.timeout = timeout
|
||||
self.default_voice = default_voice
|
||||
|
||||
@classmethod
|
||||
def from_env(cls, environ=None):
|
||||
env = os.environ if environ is None else environ
|
||||
return cls(
|
||||
voices_dir=env.get("VOICES_DIR", "/voices"),
|
||||
backend=(env.get("TTS_BACKEND") or "omni-http").strip().lower(),
|
||||
upstream=(env.get("TTS_UPSTREAM") or "http://host.containers.internal:8880").rstrip("/"),
|
||||
upstream_model=env.get("TTS_UPSTREAM_MODEL") or "voxcpm2",
|
||||
llama_bin=env.get("LLAMA_TTS_BIN") or "",
|
||||
llama_model=env.get("LLAMA_TTS_MODEL") or "",
|
||||
llama_mmproj=env.get("LLAMA_TTS_MMPROJ") or "",
|
||||
llama_lang=env.get("LLAMA_TTS_LANG") or "english",
|
||||
llama_threads=_int(env, "LLAMA_TTS_THREADS", 8),
|
||||
port=_int(env, "SHIM_PORT", 7772),
|
||||
max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
|
||||
timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
|
||||
default_voice=(env.get("DEFAULT_VOICE") or "").strip(),
|
||||
)
|
||||
|
||||
|
||||
def _int(env, name, default):
|
||||
try:
|
||||
return int(str(env.get(name, "")).strip())
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def list_voices(voices_dir):
|
||||
"""Voice names in ``voices_dir``: directories that hold a ``reference.wav``.
|
||||
|
||||
A missing or unreadable directory is an empty fleet, not an error — the
|
||||
health endpoint must still answer so the unit reports its state.
|
||||
"""
|
||||
try:
|
||||
entries = sorted(p.name for p in voices_dir.iterdir()
|
||||
if p.is_dir() and (p / "reference.wav").is_file())
|
||||
except OSError:
|
||||
return []
|
||||
return entries
|
||||
|
||||
|
||||
def resolve_voice(voices_dir, voice):
|
||||
"""``$VOICES_DIR/<voice>/reference.wav`` for a valid name, else ``None``."""
|
||||
if not isinstance(voice, str) or not VOICE_NAME_RE.fullmatch(voice):
|
||||
return None
|
||||
reference = voices_dir / voice / "reference.wav"
|
||||
return reference if reference.is_file() else None
|
||||
|
||||
|
||||
def to_mp3(wav_bytes, timeout):
|
||||
"""Re-encode WAV bytes to mp3 with the image's ``ffmpeg`` (pipes, no temp
|
||||
file: the audio never lands on disk to be left behind)."""
|
||||
argv = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0",
|
||||
"-f", "mp3", "pipe:1"]
|
||||
try:
|
||||
proc = subprocess.run(argv, input=wav_bytes, capture_output=True,
|
||||
timeout=timeout)
|
||||
except FileNotFoundError:
|
||||
raise ShimError(500, "response_format=mp3 needs the ffmpeg binary, which is not on PATH")
|
||||
except subprocess.TimeoutExpired:
|
||||
raise ShimError(502, "ffmpeg mp3 conversion timed out after %gs" % timeout)
|
||||
if proc.returncode != 0 or not proc.stdout:
|
||||
raise ShimError(502, "ffmpeg mp3 conversion failed (exit %s): %s"
|
||||
% (proc.returncode, _tail(proc.stderr)))
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def _tail(raw):
|
||||
"""Last line of a subprocess' stderr, for an error message. Text only —
|
||||
audio bytes are never echoed, and never logged."""
|
||||
text = (raw or b"").decode("utf-8", "replace").strip()
|
||||
return text.splitlines()[-1][:300] if text else "no output"
|
||||
|
||||
|
||||
def _error(message, voices=None):
|
||||
body = {"error": {"message": message, "type": "invalid_request_error"}}
|
||||
if voices is not None:
|
||||
body["error"]["available_voices"] = voices
|
||||
return body
|
||||
|
||||
|
||||
class SpeechHandler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
server_version = "frank-tts"
|
||||
sys_version = ""
|
||||
|
||||
# -- plumbing ----------------------------------------------------------
|
||||
def config(self):
|
||||
return getattr(self.server, "config", None) or Config.from_env()
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
# One prefix for the journal. Request lines and outcomes only: a
|
||||
# synthesized text, a base64 reference voice and the audio bytes
|
||||
# itself never reach stderr, in success or in failure.
|
||||
sys.stderr.write("frank-tts: %s - %s\n"
|
||||
% (self.address_string(), fmt % args))
|
||||
|
||||
def log_error(self, fmt, *args):
|
||||
# The base class routes refused requests through a separate sink; keep
|
||||
# them on the same line format so the journal reads uniformly.
|
||||
self.log_message(fmt, *args)
|
||||
|
||||
def _json(self, status, obj, head=False):
|
||||
self._bytes(status, json.dumps(obj).encode("utf-8"), "application/json", head)
|
||||
|
||||
def _bytes(self, status, data, content_type, head=False):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", content_type)
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
if not head:
|
||||
self.wfile.write(data)
|
||||
|
||||
def _drain(self):
|
||||
"""Read and discard a body this route will not answer. HTTP/1.1
|
||||
keep-alive makes an unread body the NEXT request line, so a client
|
||||
that reuses the connection would see a parse error it never sent."""
|
||||
if getattr(self, "_body_read", False):
|
||||
return # already consumed; a second read would block the response
|
||||
try:
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
except ValueError:
|
||||
return
|
||||
if length > 0:
|
||||
self.rfile.read(length)
|
||||
|
||||
def _body(self):
|
||||
"""Parse the JSON request body. Never returned to a log or an error."""
|
||||
try:
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
except ValueError:
|
||||
raise ValueError("Content-Length is not a number")
|
||||
raw = self.rfile.read(length) if length > 0 else b""
|
||||
self._body_read = True
|
||||
if not raw:
|
||||
raise ValueError("expected a JSON body")
|
||||
parsed = json.loads(raw.decode("utf-8"))
|
||||
if not isinstance(parsed, dict):
|
||||
raise ValueError("expected a JSON object")
|
||||
return parsed
|
||||
|
||||
# -- routes ------------------------------------------------------------
|
||||
def do_GET(self, head=False):
|
||||
# HEAD shares the route table: `wget --spider` (the fleet's probe
|
||||
# shape) asks for HEAD, and a 501 there would read as a dead service.
|
||||
path = urllib.parse.urlparse(self.path).path
|
||||
cfg = self.config()
|
||||
if path == "/health":
|
||||
self._json(200, {"status": "ok", "backend": cfg.backend,
|
||||
"voices": list_voices(cfg.voices_dir)}, head)
|
||||
elif path == "/v1/audio/voices":
|
||||
self._json(200, list_voices(cfg.voices_dir), head)
|
||||
else:
|
||||
self._json(404, _error("no such endpoint: %s" % path), head)
|
||||
|
||||
def do_HEAD(self):
|
||||
self.do_GET(head=True)
|
||||
|
||||
def do_POST(self):
|
||||
path = urllib.parse.urlparse(self.path).path
|
||||
if path != "/v1/audio/speech":
|
||||
self._drain()
|
||||
self._json(404, _error("no such endpoint: %s" % path))
|
||||
return
|
||||
cfg = self.config()
|
||||
try:
|
||||
body = self._body()
|
||||
except (ValueError, UnicodeDecodeError) as exc:
|
||||
self._drain()
|
||||
self._json(400, _error("malformed request body: %s" % exc))
|
||||
return
|
||||
|
||||
text = body.get("input")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
self._json(400, _error("`input` must be a non-empty string"))
|
||||
return
|
||||
if len(text) > cfg.max_chars:
|
||||
self._json(414, _error("input is %d characters, over the %d limit "
|
||||
"(SHIM_MAX_CHARS); split it or raise the limit"
|
||||
% (len(text), cfg.max_chars)))
|
||||
return
|
||||
|
||||
fmt = str(body.get("response_format") or "wav").strip().lower()
|
||||
if fmt not in ("wav", "mp3"):
|
||||
self._json(400, _error("unsupported response_format %r; this "
|
||||
"endpoint serves wav and mp3" % fmt))
|
||||
return
|
||||
|
||||
# `voice_id` is the alias the omp harness forwards for `voice`. A request
|
||||
# that names nothing takes the declared default — the harness has no
|
||||
# setting that feeds a voice, so voice-less calls are the normal shape
|
||||
# there, not a client bug.
|
||||
requested = body.get("voice") or body.get("voice_id")
|
||||
defaulted = not requested and bool(cfg.default_voice)
|
||||
voice = requested or cfg.default_voice
|
||||
reference = resolve_voice(cfg.voices_dir, voice)
|
||||
if reference is None:
|
||||
available = list_voices(cfg.voices_dir)
|
||||
listed = ", ".join(available) if available else "(none: no voice pack with a reference.wav in %s)" % cfg.voices_dir
|
||||
self._json(400, _error("unknown voice %r; available voices: %s"
|
||||
% (voice, listed), available))
|
||||
return
|
||||
|
||||
model = str(body.get("model") or cfg.upstream_model)
|
||||
try:
|
||||
if cfg.backend == "omni-http":
|
||||
audio = self._speech_http(cfg, model, text, str(voice), reference)
|
||||
elif cfg.backend == "llama-tts-cli":
|
||||
audio = self._speech_cli(cfg, text, reference)
|
||||
else:
|
||||
raise ShimError(500, "unknown TTS_BACKEND %r (expected omni-http or llama-tts-cli)" % cfg.backend)
|
||||
if fmt == "mp3":
|
||||
audio = to_mp3(audio, cfg.timeout)
|
||||
except ShimError as exc:
|
||||
self._json(exc.status, _error(exc.message))
|
||||
return
|
||||
|
||||
content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
|
||||
self._bytes(200, audio, content_type)
|
||||
self.log_message("speech voice=%s%s format=%s chars=%d -> %d bytes %s",
|
||||
voice, " (default)" if defaulted else "", fmt,
|
||||
len(text), len(audio), content_type)
|
||||
|
||||
# -- backends ----------------------------------------------------------
|
||||
def _speech_http(self, cfg, model, text, voice, reference):
|
||||
"""Forward to the upstream with the cloned voice riding along as
|
||||
base64 ``reference_audio``. The upstream is asked for wav always; mp3
|
||||
is our own re-encode on the way out."""
|
||||
try:
|
||||
payload = {
|
||||
"model": model,
|
||||
"input": text,
|
||||
"voice": voice,
|
||||
"response_format": "wav",
|
||||
"reference_audio": base64.b64encode(reference.read_bytes()).decode("ascii"),
|
||||
}
|
||||
except OSError as exc:
|
||||
raise ShimError(500, "cannot read %s: %s" % (reference, exc))
|
||||
|
||||
url = cfg.upstream + "/v1/audio/speech"
|
||||
request = urllib.request.Request(
|
||||
url, data=json.dumps(payload).encode("utf-8"),
|
||||
headers={"Content-Type": "application/json"}, method="POST")
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=cfg.timeout) as response:
|
||||
audio = response.read()
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read(512).decode("utf-8", "replace").strip()
|
||||
raise ShimError(502, "upstream %s returned HTTP %d: %s"
|
||||
% (url, exc.code, detail or "no detail"))
|
||||
except urllib.error.URLError as exc:
|
||||
raise ShimError(502, "upstream %s unreachable: %s" % (url, exc.reason))
|
||||
except (TimeoutError, OSError) as exc:
|
||||
raise ShimError(502, "upstream %s failed: %s" % (url, exc))
|
||||
if not audio:
|
||||
raise ShimError(502, "upstream %s returned no audio" % url)
|
||||
return audio
|
||||
|
||||
def _speech_cli(self, cfg, text, reference):
|
||||
"""Run the local TTS binary once, in a temp dir that is always removed
|
||||
— also on failure, or a crash-looping binary fills /tmp."""
|
||||
if not cfg.llama_bin or not cfg.llama_model:
|
||||
raise ShimError(500, "TTS_BACKEND=llama-tts-cli needs LLAMA_TTS_BIN and LLAMA_TTS_MODEL")
|
||||
work = tempfile.TemporaryDirectory(prefix="frank-tts-")
|
||||
try:
|
||||
out_path = Path(work.name) / "speech.wav"
|
||||
argv = ["nice", "-n", "19", "env", "OMP_NUM_THREADS=%d" % cfg.llama_threads,
|
||||
cfg.llama_bin, "-m", cfg.llama_model]
|
||||
if cfg.llama_mmproj:
|
||||
argv += ["-mm", cfg.llama_mmproj]
|
||||
argv += ["--tts-lang", cfg.llama_lang,
|
||||
"--tts-speaker-file", str(reference),
|
||||
"-p", text,
|
||||
"--output", str(out_path)]
|
||||
try:
|
||||
proc = subprocess.run(argv, cwd=work.name, capture_output=True,
|
||||
timeout=cfg.timeout)
|
||||
except FileNotFoundError as exc:
|
||||
raise ShimError(500, "cannot run the TTS command: %s" % exc)
|
||||
except subprocess.TimeoutExpired:
|
||||
raise ShimError(502, "TTS binary timed out after %gs" % cfg.timeout)
|
||||
if proc.returncode != 0:
|
||||
raise ShimError(502, "TTS binary failed (exit %s): %s"
|
||||
% (proc.returncode, _tail(proc.stderr)))
|
||||
if not out_path.is_file():
|
||||
raise ShimError(502, "TTS binary exited 0 but wrote no %s" % out_path.name)
|
||||
return out_path.read_bytes()
|
||||
finally:
|
||||
work.cleanup()
|
||||
|
||||
|
||||
class SpeechServer(ThreadingHTTPServer):
|
||||
"""Threads so a health probe answers while a long synthesis runs; daemon
|
||||
threads so SIGTERM is never held hostage by a stuck request."""
|
||||
|
||||
daemon_threads = True
|
||||
allow_reuse_address = True
|
||||
config = None
|
||||
|
||||
|
||||
def create_server(config=None, host="0.0.0.0", port=None):
|
||||
"""Bind the shim. ``config``/``port`` are the injectable seams: tests pass
|
||||
a Config with a temp VOICES_DIR and ``port=0`` for an ephemeral port."""
|
||||
cfg = Config.from_env() if config is None else config
|
||||
server = SpeechServer((host, cfg.port if port is None else port), SpeechHandler)
|
||||
server.config = cfg
|
||||
return server
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="frank-tts", description="OpenAI-compatible speech endpoint for cloned voice packs")
|
||||
parser.add_argument("--host", default="0.0.0.0",
|
||||
help="bind address (default: all interfaces)")
|
||||
parser.add_argument("--port", type=int, default=None,
|
||||
help="bind port (default: $SHIM_PORT, 0 = ephemeral)")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
server = create_server(host=args.host, port=args.port)
|
||||
cfg = server.config
|
||||
bound = server.server_address[1]
|
||||
print("frank-tts: backend=%s voices_dir=%s listening on http://%s:%d"
|
||||
% (cfg.backend, cfg.voices_dir, args.host, bound), flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,134 +0,0 @@
|
||||
# Quadlet unit: frank-tts-engine (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.container
|
||||
# Unit name: frank-tts-engine.service (container name: frank-tts-engine)
|
||||
#
|
||||
# WHAT IT IS. The arm A TTS engine: qwentts.cpp (pinned commit
|
||||
# 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4) built into an image by
|
||||
# dankclaw/voiceservice/Containerfile.engine, running its resident `tts-server`
|
||||
# with the 1.7 B base talker Q4_K_M and the Q8_0 12 Hz tokenizer loaded ONCE at
|
||||
# startup. It is deliberately NOT a fleet endpoint: the endpoint consumers
|
||||
# address is **frank-tts on 7772** ([[Frank TTS]]), which resolves voice names,
|
||||
# re-encodes to mp3 and owns the timeout budget. Runbook: [[Frank TTS Engine]].
|
||||
#
|
||||
# WHY THIS ENGINE (Frank's call, 2026-10-18, criterion "whatever is better for
|
||||
# live output"): server-side time-to-first-audio 94-112 ms (86-451 ms end to
|
||||
# end through the façade), streaming that stays ahead of real time at warm RTF
|
||||
# 0.55-0.67, and a run-to-run spread of +-0.01 — against the mainline CLI path's
|
||||
# 0.93-0.99 at +-0.25, which crosses 1.0 often enough to stall a live stream.
|
||||
# The A/B that decided it is "Decision: arm A for live output" in
|
||||
# [[Local TTS and Voice Cloning]].
|
||||
#
|
||||
# IMAGE. Built locally by deploy-remote.sh (build_compose_image), like
|
||||
# loki/alloy/frank-tts. `Pull=never` plus the `localhost/` prefix are required:
|
||||
# this host's quadlet path cannot resolve a locally built tag from local storage
|
||||
# otherwise. The build is a network operation (it clones the pinned commit) and
|
||||
# is skipped when the Containerfile hash label already matches.
|
||||
#
|
||||
# NETWORK, AND WHY IT PUBLISHES NOTHING. The container sits on exactly one
|
||||
# network: the private `frank-tts-engine.network` (10.89.5.0/24). The façade
|
||||
# joins the same network and dials `http://frank-tts-engine:8880` — the
|
||||
# container DNS name, on the network that owns it. There is no `PublishPort` of
|
||||
# any kind, not even `127.0.0.1:8880`, and the reason is the lesson that 502'd
|
||||
# hermes: netavark DNATs published ports only, so a host-loopback publish is not
|
||||
# a path to a peer container — the façade could never have used it, and a bridge
|
||||
# container cannot reach the host's own loopback bind either (see
|
||||
# [[Container Network Namespaces]]). A publish would therefore buy exactly one
|
||||
# thing: an extra reachable surface on an engine that has no authentication, on
|
||||
# a box whose CPU is also carrying the text endpoints. Diagnostics do not need
|
||||
# it either — `podman exec frank-tts-engine curl http://127.0.0.1:8880/health`
|
||||
# speaks to the server from inside its own namespace, which is where the address
|
||||
# is valid.
|
||||
#
|
||||
# MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs:
|
||||
# the Vulkan backend copies both tensors into device memory at load, so they
|
||||
# stop counting in the process's RSS (measured after the 2026-10-19 cutover —
|
||||
# `podman stats` / `ps -o rss`). No memory limit is set: the box has the room
|
||||
# and a limit would convert a busy moment into an OOM kill mid-utterance.
|
||||
#
|
||||
# `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan
|
||||
# ICD enumerates nothing and the server dies on device lookup after it has
|
||||
# already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the
|
||||
# thing that works here: the units podman spawns carry no supplementary groups
|
||||
# at all, and this host grants render access a different way — the udev rule
|
||||
# that makes the render nodes world-writable (same rule ComfyUI rides, see
|
||||
# dankclaw/comfyui). Measured: `id` inside this container lists only the
|
||||
# container's own group, and `tts-server` still holds renderD129.
|
||||
# The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and
|
||||
# renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a
|
||||
# re-probe can invalidate, while the directory hands the container every render
|
||||
# node and the driver picks by name.
|
||||
#
|
||||
# /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
|
||||
# the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
|
||||
# container a second way onto the GPU is surface for nothing.
|
||||
#
|
||||
# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
|
||||
# becomes known to the engine only through POST /v1/audio/voices, and the
|
||||
# registration is MEMORY ONLY — it does not survive a restart, which is exactly
|
||||
# why the warm hook below runs on every start. That hook runs INSIDE this
|
||||
# container (see ExecStartPost), so it reads the packs at /voices; `tts-server`
|
||||
# itself never opens the directory, it receives the reference audio as base64 in
|
||||
# the registration body. The façade mounts the same host directory read-only for
|
||||
# its own name list, so one directory stays the single source of a voice name.
|
||||
#
|
||||
# REGISTRY LAW, THE REASON EXECSTARTPOST EXISTS. The voice registry is memory
|
||||
# only: a restart forgets every voice, and re-registering does not warm one —
|
||||
# only synthesizing does — so a boot without this hook leaves the façade
|
||||
# answering 400 for every name. The per-voice cold penalty is real but small, and
|
||||
# not the trial's 6.2-8.0 s: measured on the built container 2026-10-18 against an
|
||||
# empty registry, the first synthesis of a freshly registered voice cost 1,883 ms
|
||||
# against 1,226-1,283 ms for the requests after it. The warm pass runs serially
|
||||
# (two short syntheses per voice, ~5 s each) and `TimeoutStartSec=900` keeps a growing library from
|
||||
# tripping systemd's default 90 s start deadline. The leading `-` on
|
||||
# ExecStartPost is load-bearing: with `Restart=always`, a warmer that exits
|
||||
# non-zero (a pack missing its reference.wav, the engine still loading past the
|
||||
# warmer's own patience) would restart the whole container and crash-loop the
|
||||
# endpoint forever. A failed warm must be a journal line, not a restart.
|
||||
#
|
||||
# The hook is `podman exec`, not a host script, because there is no address the
|
||||
# host could dial: this container publishes nothing (above). Exec-ing into it
|
||||
# also means the warmer inherits this unit's EnvironmentFile, so every knob it
|
||||
# has lives in scripts/frank-tts-engine.env beside the engine's own, and it sees
|
||||
# the same /voices and 127.0.0.1:8880 the engine does.
|
||||
#
|
||||
# HEALTH. `GET /health` answers `{"status":"ok"}` from the HTTP thread, not the
|
||||
# synthesis queue, so a busy engine still reads healthy and a dead one does not.
|
||||
# HealthStartPeriod covers the first model load. The command is argument-only
|
||||
# and needs /bin/sh — the generator wraps HealthCmd as CMD-SHELL — so the image
|
||||
# installs curl (see the opposite case in dankclaw/loki, whose distroless base
|
||||
# has no shell).
|
||||
|
||||
[Unit]
|
||||
Description=frank-tts-engine (arm A qwentts.cpp TTS engine, internal :8880)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=frank-tts-engine
|
||||
Image=localhost/franksdank/frank-tts-engine:latest
|
||||
Pull=never
|
||||
Network=frank-tts-engine.network
|
||||
# No PublishPort, at all: the only reachable address this server has is the
|
||||
# bridge address the façade resolves by name. Diagnostics use `podman exec`.
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
|
||||
Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
|
||||
Volume=/home/dank/voices:/voices:ro
|
||||
# Render access for the iGPU (see MEMORY / THE RENDER NODE above).
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
HealthCmd=curl -fsS http://127.0.0.1:8880/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=120s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
# Model load (tens of seconds) plus two short syntheses per registered voice.
|
||||
TimeoutStartSec=900
|
||||
ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,52 +0,0 @@
|
||||
# Quadlet network: the private bridge of the voice pair (rootless).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.network
|
||||
# Unit name: frank-tts-engine-network.service (podman network name:
|
||||
# systemd-frank-tts-engine — quadlet prefixes "systemd-", so probe the runtime
|
||||
# name, never the filename; see [[Container Network Namespaces]]).
|
||||
#
|
||||
# WHY A PRIVATE BRIDGE FOR A PAUSE THAT LOOKS PUBLIC-FACING. frank-tts (:7772,
|
||||
# published, the one endpoint consumers address) and frank-tts-engine (:8880,
|
||||
# never published, the arm A qwentts.cpp server behind it) could both have
|
||||
# stayed on the fleet-wide dankclaw_default and resolved each other by name.
|
||||
# They are split off anyway, for two reasons:
|
||||
#
|
||||
# 1. THE HOST-LOOPBACK TRAP. The phase-1 shape in `scripts/frank-tts.env` still
|
||||
# names `host.containers.internal:8880` as the upstream, on the assumption the
|
||||
# engine runs on the host and publishes `127.0.0.1:8880`. That cannot work:
|
||||
# netavark DNATs published ports only, a published port is not on any podman
|
||||
# bridge, and a container cannot reach the host's own loopback bind — the same
|
||||
# shape that 502'd hermes, and the reason the signal-relay hop needs a host LAN
|
||||
# bind instead ([[Container Network Namespaces]], [[Homepage Status Probes]]).
|
||||
# Putting both containers on one committed network removes the guess: the
|
||||
# façade dials the engine's container DNS name on the network that owns it, and
|
||||
# the engine publishes nothing at all.
|
||||
#
|
||||
# 2. THE ENGINE HAS NO AUTH AND NO QUEUE LIMIT WORTH THE NAME. `tts-server`
|
||||
# answers anyone who can open a socket, and at `--max-batch 1` a second
|
||||
# caller simply waits behind the first. Left on dankclaw_default every
|
||||
# container on that net — open-webui, hermes, the *arr proxies — could drive
|
||||
# a CPU synthesiser that occupies every other core on a box that also serves
|
||||
# two text endpoints. On this bridge the only dialer is frank-tts: the engine
|
||||
# publishes no port, so not even the host has an address to reach it at, and
|
||||
# diagnostics go through `podman exec` instead.
|
||||
#
|
||||
# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
|
||||
# per-container userspace helper left a container that accepted TCP and reset
|
||||
# every request for two days, logging nothing). A bridge has no such helper.
|
||||
#
|
||||
# Subnet registry on dankclaw (rootless):
|
||||
# 10.88.0.0/16 default "podman" bridge
|
||||
# 10.89.0.0/24 gitea (systemd-gitea)
|
||||
# 10.89.1.0/24 compose stack (dankclaw_default)
|
||||
# 10.89.2.0/24 searxng (systemd-searxng)
|
||||
# 10.89.3.0/24 open-webui (systemd-open-webui)
|
||||
# 10.89.4.0/24 open-notebook (systemd-open-notebook)
|
||||
# 10.89.5.0/24 voice pair (systemd-frank-tts-engine) <- this file
|
||||
# Pinned so the two container IPs stay deterministic across reboots, like the
|
||||
# other pinned nets; 10.89.5.0/24 was the next unclaimed /24 in this registry.
|
||||
|
||||
[Network]
|
||||
Driver=bridge
|
||||
Subnet=10.89.5.0/24
|
||||
Gateway=10.89.5.1
|
||||
@@ -1,84 +0,0 @@
|
||||
# Quadlet unit: frank-tts (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts.container
|
||||
# Unit name: frank-tts.service (container name: frank-tts)
|
||||
#
|
||||
# WHAT IT IS. The fleet's one OpenAI-compatible speech endpoint: consumers
|
||||
# (omp, OpenWebUI, open-notebook) POST /v1/audio/speech with voice=<name> and
|
||||
# get audio in that cloned voice. A voice is a directory under
|
||||
# /home/dank/voices holding reference.wav; the shim refuses any name without
|
||||
# one (400 + the list that does exist) rather than narrating in a fallback
|
||||
# voice nobody chose. The engine behind it is a separate decision —
|
||||
# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to the
|
||||
# OpenAI-shaped engine (`frank-tts-engine`, [[Frank TTS Engine]]) and running a
|
||||
# local CLI binary per request.
|
||||
#
|
||||
# IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
|
||||
# deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
|
||||
# pinned by digest + ffmpeg, app.py copied in, running as uid 10000.
|
||||
# Pull=never and the localhost/ prefix are required — this host's quadlet
|
||||
# path cannot resolve a locally built tag from local storage otherwise.
|
||||
#
|
||||
# NETWORK. TWO attached networks, and the order is the one the open-notebook
|
||||
# pair uses: `dankclaw_default` first (the default route, and where consumers
|
||||
# live — they reach the published 7772, and open-notebook dials
|
||||
# `http://frank-tts:7772/v1` by container name), then the voice pair's private
|
||||
# bridge, `frank-tts-engine.network`, where the arm A engine lives. The shim's
|
||||
# upstream is therefore the engine's container DNS name
|
||||
# (`http://frank-tts-engine:8880`, scripts/frank-tts.env) and NOT
|
||||
# host.containers.internal: netavark DNATs published ports only, so a bridge
|
||||
# container cannot reach a host-loopback bind — the shape that 502'd hermes —
|
||||
# and the engine publishes nothing at all. That file MUST be installed beside
|
||||
# this unit (the engine's handler does it; the façade handler re-installs it for
|
||||
# `--only frank-tts`), because with a referenced Network= file missing the v1
|
||||
# generator refuses to produce this container unit at all.
|
||||
#
|
||||
# ORDERING. Wants+After on frank-tts-engine.service, not Requires: the façade
|
||||
# has a second backend (`TTS_BACKEND=llama-tts-cli`) that needs no upstream, and
|
||||
# an engine that will not start must not take this endpoint's unit down with it.
|
||||
# When both boot, this comes up after the engine, whose ExecStartPost warm pass
|
||||
# is part of its start — so the first request of the day meets a REGISTERED voice
|
||||
# (the registry is memory-only) rather than a 400, and skips the ~0.6 s penalty
|
||||
# the first synthesis of an unwarmed voice costs.
|
||||
#
|
||||
# VOICES. /home/dank/voices is host state, never repo state, and is mounted
|
||||
# READ-ONLY: this endpoint has no business writing voice packs. Because the
|
||||
# container runs as uid 10000, the packs must be world-readable (644 files,
|
||||
# 755 directories), which is how the pack builder writes them.
|
||||
#
|
||||
# NO GPU. The shim is a façade: audio conversion via ffmpeg on the CPU and,
|
||||
# for the CLI backend, whatever CPU threads LLAMA_TTS_THREADS allows, at nice
|
||||
# 19. No /dev/kfd, no /dev/dri, no keep-groups — unlike comfyui, whose unit
|
||||
# carries them deliberately.
|
||||
[Unit]
|
||||
Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
|
||||
After=network-online.target frank-tts-engine.service
|
||||
Wants=network-online.target
|
||||
Wants=frank-tts-engine.service
|
||||
|
||||
[Container]
|
||||
ContainerName=frank-tts
|
||||
Image=localhost/franksdank/frank-tts:latest
|
||||
Pull=never
|
||||
Network=dankclaw_default
|
||||
Network=frank-tts-engine.network
|
||||
PublishPort=7772:7772
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
|
||||
Volume=/home/dank/voices:/voices:ro
|
||||
# The probe is a probe, not a voice test: /health answers 200 with the backend
|
||||
# and the resolved voice list, so a mounted-but-empty /voices still reads
|
||||
# healthy while a dead app does not. busybox wget ships in the alpine base;
|
||||
# keep the command argument-only — the podman 5.x generator splits HealthCmd
|
||||
# on spaces (see the comfyui unit for the quoting trap).
|
||||
HealthCmd=wget --spider -q http://127.0.0.1:7772/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,124 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tts-engine-start.sh — the entrypoint of the frank-tts-engine image
|
||||
# (dankclaw/voiceservice/Containerfile.engine). It exists for one reason: the
|
||||
# knobs of the arm A engine are command-line flags, while the fleet's knobs
|
||||
# live in scripts/frank-tts-engine.env and reach the container through
|
||||
# EnvironmentFile=. This is the translation layer, and nothing else — no
|
||||
# supervision, no retries, no logging of its own. PID 1 stays the server, so
|
||||
# `podman stop` and the unit's Restart=always drive it directly.
|
||||
#
|
||||
# THE FLAG THAT MUST STAY ABSENT:
|
||||
#
|
||||
# --threads / -t does not exist in this build. src/backend.h computes the
|
||||
# CPU thread count as std::thread::hardware_concurrency()/2
|
||||
# (the comment there: GEMM shares SIMD units across
|
||||
# hyperthreads), and there is no flag to override it. So a
|
||||
# CPU affinity pin (CPUQuota=, taskset, a cpuset) silently
|
||||
# changes the engine's thread count instead of just its
|
||||
# placement — pinning is deliberately not done anywhere in
|
||||
# this stack. See the env file header for the same warning
|
||||
# aimed at the operator.
|
||||
#
|
||||
# --device IS here, but conditionally: it reaches the binary only when
|
||||
# QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
|
||||
# takes its own default device, which is CPU because the CPU backend is always
|
||||
# built. One line in scripts/frank-tts-engine.env moves the endpoint between the
|
||||
# two shapes, with no rebuild. What must never reach argv is a device NAME the
|
||||
# loader does not enumerate: that is not refused by the argument parser, it dies
|
||||
# during device lookup with the loader's own error, so the journal reads as a
|
||||
# load that failed rather than a configuration refusal.
|
||||
#
|
||||
# Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
|
||||
# the Q8_0 tokenizer, one request at a time, on the private bridge, on the
|
||||
# gfx1151 iGPU. An empty environment still boots that shape.
|
||||
set -euo pipefail
|
||||
|
||||
# The binary path. Overridable for one reason: the bats seam runs a stub
|
||||
# instead of a 1.2 GiB model. Operators never set it — the image ships exactly
|
||||
# one binary at /app/tts-server.
|
||||
: "${QWENTTS_BIN:=/app/tts-server}"
|
||||
|
||||
# Paths are CONTAINER paths (the host directories they come from are named in
|
||||
# scripts/frank-tts-engine.env and mounted by the Quadlet unit).
|
||||
: "${QWENTTS_MODEL:=/models/qwen-talker-1.7b-base-Q4_K_M.gguf}"
|
||||
: "${QWENTTS_CODEC:=/models/qwen-tokenizer-12hz-Q8_0.gguf}"
|
||||
|
||||
# Reported by GET /v1/models, and what scripts/frank-tts.env's
|
||||
# TTS_UPSTREAM_MODEL names. A model id, not a filename: the file name would
|
||||
# leak a quantization choice into every consumer's config.
|
||||
: "${QWENTTS_ALIAS:=frank-tts-engine}"
|
||||
|
||||
# 0.0.0.0 inside the container is not "public". tts-server defaults to
|
||||
# 127.0.0.1 (its own usage prints `--host <ip> Listen address (default:
|
||||
# 127.0.0.1)`), and inside a container netns a loopback bind is reachable by
|
||||
# nothing outside that namespace — the façade dials this container by its bridge
|
||||
# address on the private voice network, which is why the address has to be
|
||||
# broad. The unit PUBLISHES NOTHING (see
|
||||
# dankclaw/voiceservice/quadlet/frank-tts-engine.container: no PublishPort at
|
||||
# all, not even 127.0.0.1:8880), so the only namespaces that can reach
|
||||
# 0.0.0.0:8880 are that bridge and this container's own loopback — and its own
|
||||
# loopback is exactly where the boot warmer and the health probe speak.
|
||||
: "${QWENTTS_HOST:=0.0.0.0}"
|
||||
: "${QWENTTS_PORT:=8880}"
|
||||
|
||||
# The language label used when a request omits one. `auto` is the upstream
|
||||
# default and what the trial ran.
|
||||
: "${QWENTTS_LANG:=auto}"
|
||||
|
||||
# One request at a time. The server-side queue is a serialised FIFO at this
|
||||
# setting, which is what keeps the per-voice timings predictable. A second
|
||||
# concurrent synthesis shares the device with the first — and on this box that
|
||||
# device is also carrying frank-dragoon — so it buys no throughput and it
|
||||
# doubles the wait for the caller behind it. The engine's own default is also 1;
|
||||
# written here because it is a decision, not an accident.
|
||||
: "${QWENTTS_MAX_BATCH:=1}"
|
||||
|
||||
# The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
|
||||
# the gfx1151 iGPU on this host. An index would follow whatever order the
|
||||
# loader's device list returns; a name says which device this endpoint means.
|
||||
# The container sees exactly one Vulkan device because the image carries the
|
||||
# Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
|
||||
#
|
||||
# EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
|
||||
# the binary picks its own default, and the CPU backend is always built — which
|
||||
# is why moving this endpoint between the iGPU and the CPU is one line in
|
||||
# scripts/frank-tts-engine.env and not a rebuild.
|
||||
#
|
||||
# That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
|
||||
# fires on a set-but-empty variable as well as an unset one, so the rollback line
|
||||
# `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
|
||||
# on the shared GPU with no way out short of a rebuild. Only UNSET takes the
|
||||
# default.
|
||||
if [[ ! -v QWENTTS_DEVICE ]]; then
|
||||
QWENTTS_DEVICE=Vulkan0
|
||||
fi
|
||||
|
||||
# Fail loudly BEFORE binding, so the journal says which artifact is missing
|
||||
# instead of the unit reporting a container that starts and dies. This is the
|
||||
# self-healing path: Restart=always re-runs it every RestartSec, and the same
|
||||
# sentence repeats until the operator fetches the weights
|
||||
# (scripts/download-frank-tts-engine-qwentts.sh).
|
||||
for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
|
||||
if [[ ! -r "$artifact" ]]; then
|
||||
echo "frank-tts-engine: cannot read ${artifact}; run scripts/download-frank-tts-engine-qwentts.sh on the host (the /models mount is read-only and holds the two GGUF files)" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
ARGS=(
|
||||
--model "$QWENTTS_MODEL"
|
||||
--codec "$QWENTTS_CODEC"
|
||||
--alias "$QWENTTS_ALIAS"
|
||||
--host "$QWENTTS_HOST"
|
||||
--port "$QWENTTS_PORT"
|
||||
--lang "$QWENTTS_LANG"
|
||||
--max-batch "$QWENTTS_MAX_BATCH"
|
||||
)
|
||||
|
||||
# Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
|
||||
# because "the rollback shape" is argv identical to the pre-iGPU endpoint.
|
||||
if [[ -n "$QWENTTS_DEVICE" ]]; then
|
||||
ARGS+=(--device "$QWENTTS_DEVICE")
|
||||
fi
|
||||
|
||||
exec "$QWENTTS_BIN" "${ARGS[@]}"
|
||||
@@ -1,222 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tts-engine-warm.sh — the boot warmer of the arm A TTS engine, shipped inside
|
||||
# the frank-tts-engine image at /app/warm-voices.sh
|
||||
# (dankclaw/voiceservice/Containerfile.engine).
|
||||
#
|
||||
# WHO RUNS IT. Two callers, same binary:
|
||||
#
|
||||
# 1. The Quadlet unit, as
|
||||
# `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
|
||||
# — after every start of the container, including every boot.
|
||||
# `podman exec` rather than a host script because the engine publishes no
|
||||
# port at all, so there is no address the host could dial; and because an
|
||||
# exec inherits the unit's EnvironmentFile, which is what puts every knob of
|
||||
# this script in scripts/frank-tts-engine.env next to the engine's own.
|
||||
# 2. scripts/deploy-remote.sh's deploy_frank_tts_engine, once per deploy, so a
|
||||
# voice that did not warm becomes a deploy result instead of only a journal
|
||||
# line (the unit's `-` prefix deliberately cannot fail the unit).
|
||||
#
|
||||
# WHY IT EXISTS AT ALL (the registry law). The voice registry is MEMORY ONLY: a
|
||||
# restart forgets every voice, and re-registering a voice does not warm it — only
|
||||
# synthesizing does. A boot that registers nothing therefore leaves frank-tts
|
||||
# answering 400 for every name, so this script registers every pack it finds. The
|
||||
# per-voice cold penalty is real but small, and smaller than the trial reported:
|
||||
# measured on the deployed build 2026-10-18 against an empty registry, the FIRST
|
||||
# synthesis of a freshly registered voice cost 1,883 ms against 1,226-1,283 ms for
|
||||
# the requests after it — about 0.6 s, not the trial's 6.2-8.0 s. Each voice gets
|
||||
# two identical requests so the journal carries both numbers, `voice <name>
|
||||
# cold=…ms warm=…ms`; cold ≈ warm is the expected shape, and a large gap is what
|
||||
# would mean the snapshot path is broken.
|
||||
#
|
||||
# THE VOICES. /voices — the host directory /home/dank/voices mounted read-only by
|
||||
# the unit, the same packs frank-tts mounts for its own name list. A pack is a
|
||||
# directory that contains reference.wav; everything else in it (manifest.json, the
|
||||
# engine's reference.spk / reference.rvq extraction, a 16 kHz variant) is not read
|
||||
# here. The registration payload is `{"name": <dir>, "wav_b64": <base64 of
|
||||
# reference.wav>}` with NO ref_text, which selects the embedding-only (x-vector)
|
||||
# voice — the shipped default, warm RTF ~0.57-0.63 against ICL's 0.67, and the one
|
||||
# this fleet can produce honestly because it has no trustworthy transcript for the
|
||||
# audition crops. Adding ref_text (with the codes) would flip that voice to
|
||||
# in-context-learning mode: better fidelity on a good transcript, slower, and only
|
||||
# worth it for batch work like a podcast render. That is a per-voice opt-in this
|
||||
# script deliberately does NOT exercise, and the door stays open: a pack that
|
||||
# wants ICL needs its text in the pack plus one key in the payload below.
|
||||
#
|
||||
# A lone `spk_b64` is a 400 (src/tts-server.h:480 — it is `wav_b64` or BOTH
|
||||
# `spk_b64` and `rvq_b64`), which is why the WAV travels and the .spk stays home.
|
||||
#
|
||||
# NOTHING HERE IS A SECRET, and nothing is logged that could be: the audio bytes
|
||||
# are discarded, the synthesized text comes from the env file, and the reference
|
||||
# clip is base64'd into a request body that is written to a temporary file and
|
||||
# handed to curl as `@file` — never echoed, and never placed on a command line,
|
||||
# because of the size law explained at the mktemp below.
|
||||
set -euo pipefail
|
||||
|
||||
# The engine's own address, from inside its namespace. Loopback is valid HERE:
|
||||
# this script runs in the container, next to the server. There is deliberately no
|
||||
# host-visible address to use instead — the unit publishes nothing, because a
|
||||
# host-loopback publish would be unreachable from the façade anyway (netavark
|
||||
# DNATs published ports only) while adding a surface on an unauthenticated
|
||||
# server.
|
||||
: "${ENGINE_URL:=http://127.0.0.1:8880}"
|
||||
|
||||
# Container path; the host directory behind it is named in the Quadlet unit.
|
||||
: "${VOICES_DIR:=/voices}"
|
||||
|
||||
# How long to wait for the engine to answer /health. Model load dominates: the
|
||||
# two GGUFs (1.2 GiB + 291 MB) are read and the context built before the HTTP
|
||||
# thread binds, and a boot races every other unit on this box.
|
||||
: "${HEALTH_WAIT:=300}"
|
||||
|
||||
# One short sentence, long enough to exercise the codec snapshot and short enough
|
||||
# that warming a library of voices is seconds, not minutes. Never varied per run:
|
||||
# the cold/warm numbers in the journal should mean the same thing every boot. The
|
||||
# synthesis asks for `wav`, the buffered path — the streamed path (`pcm`) is
|
||||
# different audio at the same seed, and `wav` is also what frank-tts asks for.
|
||||
: "${WARM_TEXT:=Warm up, one two three.}"
|
||||
|
||||
# Per-request ceiling. A cold voice is 6-8 s by measurement; anything approaching
|
||||
# this is a stalled engine and should be reported rather than waited on forever —
|
||||
# systemd counts the whole warm pass against the unit's start deadline.
|
||||
: "${REQUEST_TIMEOUT:=120}"
|
||||
|
||||
# The warm text goes into a JSON string literal by interpolation. Two steps make
|
||||
# whatever the two readers of scripts/frank-tts-engine.env deliver arrive as the
|
||||
# sentence itself:
|
||||
#
|
||||
# 1. Unwrap ONE shell-style quoting layer. bash (`set -a; source`) strips the
|
||||
# quotes around a value with spaces; podman's `--env-file` does NOT — measured
|
||||
# 2026-10-18 against the image's pinned base, `WARM_TEXT="…"` reaches the
|
||||
# container WITH its quotes. Unwrapped only when the whole value is exactly one
|
||||
# quoted string (starts AND ends with `"` and holds no quote inside), so a
|
||||
# sentence that legitimately contains quotes is never mangled.
|
||||
# 2. Escape the two characters that would corrupt or end a JSON string: a
|
||||
# backslash, and a double quote.
|
||||
#
|
||||
# Without step 1 the body is `{"input":""Warm up…"",…}`, the engine's JSON parser
|
||||
# rejects it, every voice stays cold, and the boot still reads healthy. Voice NAMES
|
||||
# need neither step: the character class checked below admits no quote, no
|
||||
# backslash and no control byte.
|
||||
warm_text="$WARM_TEXT"
|
||||
if (( ${#warm_text} >= 2 )) && [[ "$warm_text" == \"*\" ]]; then
|
||||
inner="${warm_text:1:${#warm_text}-2}"
|
||||
[[ "$inner" != *'"'* ]] && warm_text="$inner"
|
||||
fi
|
||||
warm_json="${warm_text//\\/\\\\}"
|
||||
warm_json="${warm_json//\"/\\\"}"
|
||||
|
||||
# The registration body is a FILE, never an argument. The kernel caps a single
|
||||
# command-line string at 32 pages — MAX_ARG_STRLEN = 131,072 bytes — and a shipped
|
||||
# reference clip base64's to about 1.27 MB, so passing the body as one argument
|
||||
# fails with E2BIG before curl is even executed. The first real boot of the unit
|
||||
# (2026-10-18, the three shipped packs) proved it end to end: `/usr/bin/curl:
|
||||
# Argument list too long` for every voice, reported as the misleading
|
||||
# "registration failed (HTTP no response)". `mktemp` lands in the container's
|
||||
# /tmp as the unprivileged `tts` user; one file for the whole pass, overwritten
|
||||
# per voice, removed on any exit — it is container scratch, not host state.
|
||||
body_file="$(mktemp -t frank-tts-warm.XXXXXX)"
|
||||
trap 'rm -f "$body_file"' EXIT
|
||||
|
||||
fail=0
|
||||
log() { printf 'frank-tts-engine-warm: %s\n' "$*"; }
|
||||
|
||||
# Wait for /health. It answers from the HTTP thread, so a 200 means the server is
|
||||
# listening; it says nothing about voices, which is what the rest of this script
|
||||
# is for.
|
||||
ready=0
|
||||
for ((i = 1; i <= HEALTH_WAIT; i++)); do
|
||||
if curl -fsS --max-time 5 "${ENGINE_URL}/health" >/dev/null 2>&1; then
|
||||
ready=1
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if (( ready == 0 )); then
|
||||
log "engine did not answer ${ENGINE_URL}/health within ${HEALTH_WAIT}s; nothing warmed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! -d "$VOICES_DIR" ]]; then
|
||||
log "no voice directory at ${VOICES_DIR}; nothing to warm (frank-tts will answer 400 with an empty voice list until packs are built)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
warmed=0
|
||||
for dir in "$VOICES_DIR"/*/; do
|
||||
[[ -d "$dir" ]] || continue
|
||||
name="${dir%/}"
|
||||
name="${name##*/}"
|
||||
if [[ ! -f "${dir}reference.wav" ]]; then
|
||||
continue # not a pack; frank-tts refuses this name too
|
||||
fi
|
||||
# The engine lowercases every voice name (tts_voice_name) while frank-tts
|
||||
# matches the directory name exactly, so a mixed-case pack would work by
|
||||
# accident on one side and not the other. Refuse it loudly instead. Pack
|
||||
# names are also JSON string content here, so the character class is what
|
||||
# keeps a directory name from writing structure into the request body.
|
||||
if [[ ! "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
|
||||
log "voice '${name}': name must be lowercase [a-z0-9._-] (the engine lowercases names; the shim does not) — skipped"
|
||||
fail=1
|
||||
continue
|
||||
fi
|
||||
|
||||
# The audio enters through a redirection rather than a command substitution:
|
||||
# the file is several hundred KB, and both a shell variable and an argument of
|
||||
# that size are what this script used to die on.
|
||||
{
|
||||
printf '{"name":"%s","wav_b64":"' "$name"
|
||||
base64 -w0 "${dir}reference.wav"
|
||||
printf '"}'
|
||||
} > "$body_file"
|
||||
if ! code="$(curl -sS --max-time "$REQUEST_TIMEOUT" -o /dev/null -w '%{http_code}' \
|
||||
-H 'Content-Type: application/json' --data-binary "@${body_file}" \
|
||||
"${ENGINE_URL}/v1/audio/voices")" || [[ "$code" != "200" ]]; then
|
||||
log "voice '${name}': registration failed (HTTP ${code:-no response}) — the shim will answer 400 for this name"
|
||||
fail=1
|
||||
continue
|
||||
fi
|
||||
|
||||
# The cold pass. Its output is discarded; the point is the codec snapshot it
|
||||
# leaves inside the engine.
|
||||
start=$(date +%s%3N)
|
||||
if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
|
||||
"${ENGINE_URL}/v1/audio/speech"; then
|
||||
log "voice '${name}': registered, but the warm synthesis failed — first live request stays cold"
|
||||
fail=1
|
||||
continue
|
||||
fi
|
||||
cold=$(( $(date +%s%3N) - start ))
|
||||
|
||||
# The warm pass, same request. This is the number the fleet actually
|
||||
# experiences, and it is what "the second request answers under a second" is
|
||||
# measured against.
|
||||
start=$(date +%s%3N)
|
||||
if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
|
||||
"${ENGINE_URL}/v1/audio/speech"; then
|
||||
log "voice '${name}': warm synthesis failed after a cold one that worked"
|
||||
fail=1
|
||||
continue
|
||||
fi
|
||||
warm=$(( $(date +%s%3N) - start ))
|
||||
|
||||
log "voice ${name} cold=${cold}ms warm=${warm}ms"
|
||||
warmed=$(( warmed + 1 ))
|
||||
done
|
||||
|
||||
# Failure first, emptiness second, and not the other way round: a pass where
|
||||
# every voice failed also has `warmed == 0`, and reading that as "nothing to do,
|
||||
# success" is how an endpoint that answers 400 for every voice name gets deployed
|
||||
# on a Friday.
|
||||
if (( fail )); then
|
||||
log "${warmed} voice(s) warm, at least one failed (see the lines above)"
|
||||
exit 1
|
||||
fi
|
||||
if (( warmed == 0 )); then
|
||||
log "no voice with a reference.wav under ${VOICES_DIR}; engine is up but speaks nothing"
|
||||
exit 0
|
||||
fi
|
||||
log "${warmed} voice(s) registered and warm"
|
||||
@@ -1,156 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""HTTP front door for the frank-medium single-GPU replicas.
|
||||
|
||||
Why two replicas instead of TP=2: one vLLM process spanning both R9700s needs
|
||||
RCCL collectives, and RCCL's kernels require PCIe atomics on every
|
||||
participating GPU. The second card sits behind chipset bridges advertising
|
||||
`AtomicOpsCap: Routing-`, which no software can enable (see the runbook), so
|
||||
no collective can launch on it. Two single-GPU replicas use both cards; this
|
||||
service keeps the fleet contract of one endpoint on :7770, streaming included.
|
||||
|
||||
Routing decisions live in routing.py — spread single-turn requests, pin
|
||||
continuations to the replica whose prefix cache holds their context. This file
|
||||
only moves bytes: parse enough of the request to route it, forward it verbatim,
|
||||
stream the answer back, and report replica health on /health.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from contextlib import suppress
|
||||
from aiohttp import ClientSession, ClientTimeout, web
|
||||
|
||||
import routing
|
||||
|
||||
PORT = int(os.environ.get("PORT", "7770"))
|
||||
HEALTH_TTL = float(os.environ.get("HEALTH_TTL", "10"))
|
||||
# Hop-by-hop headers must not be forwarded (RFC 9110 §7.6.1) — forwarding
|
||||
# transfer-encoding or content-length of a rewritten body breaks clients.
|
||||
HOP_BY_HOP = {
|
||||
"host", "content-length", "transfer-encoding", "connection",
|
||||
"keep-alive", "proxy-authenticate", "proxy-authorization", "te",
|
||||
"trailers", "upgrade",
|
||||
}
|
||||
|
||||
UPSTREAMS = routing.UPSTREAMS
|
||||
# Replica health, refreshed by the watcher. Module state rather than aiohttp app
|
||||
# state: the application is frozen once the site is up.
|
||||
_UP: list[bool] = [True] * len(UPSTREAMS)
|
||||
|
||||
|
||||
async def health(session: ClientSession) -> web.Response:
|
||||
results = await asyncio.gather(
|
||||
*(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
|
||||
return_exceptions=True,
|
||||
)
|
||||
lines, up = [], 0
|
||||
for url, res in zip(UPSTREAMS, results):
|
||||
if isinstance(res, Exception) or res.status != 200:
|
||||
lines.append(f"{url} DOWN")
|
||||
else:
|
||||
up += 1
|
||||
lines.append(f"{url} UP")
|
||||
return web.Response(status=200 if up else 503, text="\n".join(lines) + "\n")
|
||||
|
||||
|
||||
async def watch_health() -> None:
|
||||
"""Refresh the replica up/down table so routing never targets a dead one.
|
||||
|
||||
One HTTP request per replica per interval, in exchange for never handing a
|
||||
client a 502 after a restart.
|
||||
"""
|
||||
global _UP
|
||||
session = ClientSession()
|
||||
try:
|
||||
while True:
|
||||
res = await asyncio.gather(
|
||||
*(session.get(u + "/health", timeout=ClientTimeout(total=5)) for u in UPSTREAMS),
|
||||
return_exceptions=True,
|
||||
)
|
||||
_UP = [not isinstance(r, Exception) and r.status == 200 for r in res]
|
||||
await asyncio.sleep(HEALTH_TTL)
|
||||
finally:
|
||||
await session.close()
|
||||
|
||||
|
||||
async def proxy(request: web.Request) -> web.StreamResponse:
|
||||
session: ClientSession = request.app["session"]
|
||||
if request.path == "/health":
|
||||
return await health(session)
|
||||
|
||||
body = await request.read()
|
||||
headers = {k: v for k, v in request.headers.items() if k.lower() not in HOP_BY_HOP}
|
||||
|
||||
# Fail over before anything is written to the client. A replica that
|
||||
# restarts is invisible to the health watcher for up to HEALTH_TTL, and a
|
||||
# deploy restarts both replicas — without this retry a 10 s window of
|
||||
# requests would 502 on a service that has a live card the whole time.
|
||||
# vLLM's OpenAI endpoints are stateless per request, so re-sending a
|
||||
# request that never reached an engine is safe; a request that failed
|
||||
# mid-stream is not retried (that path is below).
|
||||
tried: set[int] = set()
|
||||
upstream = None
|
||||
last_error = "no healthy frank-medium replica"
|
||||
while len(tried) < len(UPSTREAMS):
|
||||
idx = routing.route(body, request.remote, _UP)
|
||||
if idx < 0 or idx in tried:
|
||||
break
|
||||
tried.add(idx)
|
||||
target = UPSTREAMS[idx] + request.path_qs
|
||||
try:
|
||||
upstream = await session.request(
|
||||
request.method, target, headers=headers, data=body or None,
|
||||
timeout=ClientTimeout(total=None, sock_connect=10, sock_read=3600),
|
||||
)
|
||||
break
|
||||
except Exception as exc:
|
||||
# Demote at once: the watcher will confirm, but every request until
|
||||
# then should skip this replica rather than pay the same timeout.
|
||||
# Logged because a silent failover hides a card that is down.
|
||||
_UP[idx] = False
|
||||
last_error = f"{target} unreachable: {exc}"
|
||||
print(f"proxy: failover — {target} unreachable ({exc}); retrying "
|
||||
f"another replica", flush=True)
|
||||
if upstream is None:
|
||||
# 503 when no replica was even reachable through the routing table,
|
||||
# 502 when replicas were tried and all of them failed.
|
||||
status = 502 if tried else 503
|
||||
return web.Response(status=status, text=f"proxy: {last_error}\n")
|
||||
|
||||
out = web.StreamResponse(status=upstream.status, reason=upstream.reason)
|
||||
for k, v in upstream.headers.items():
|
||||
if k.lower() not in HOP_BY_HOP:
|
||||
out.headers[k] = v
|
||||
await out.prepare(request)
|
||||
try:
|
||||
async for chunk in upstream.content.iter_any():
|
||||
await out.write(chunk)
|
||||
except Exception:
|
||||
# The client went away mid-stream (an aborted generation). Nothing to
|
||||
# report: the engine sees its own cancellation.
|
||||
pass
|
||||
finally:
|
||||
upstream.release()
|
||||
with suppress(Exception):
|
||||
await out.write_eof()
|
||||
return out
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
app = web.Application(client_max_size=1024 ** 3)
|
||||
app["session"] = ClientSession()
|
||||
app.router.add_route("*", "/{tail:.*}", proxy)
|
||||
runner = web.AppRunner(app)
|
||||
await runner.setup()
|
||||
await web.TCPSite(runner, "0.0.0.0", PORT).start()
|
||||
print(f"proxy: :{PORT} -> {', '.join(UPSTREAMS)} routing={routing.ROUTING} "
|
||||
f"key={routing.AFFINITY_BYTES}B known_max={routing.KNOWN_MAX}", flush=True)
|
||||
watcher = asyncio.create_task(watch_health())
|
||||
with suppress(asyncio.CancelledError):
|
||||
await asyncio.gather(watcher)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
with suppress(KeyboardInterrupt):
|
||||
asyncio.run(main())
|
||||
@@ -1,98 +0,0 @@
|
||||
# Quadlet unit: frank-medium replica A (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/frank-medium-a.container
|
||||
# Unit name: frank-medium-a.service (container name: frank-medium-a)
|
||||
#
|
||||
# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
|
||||
# tensor-parallel process across both R9700s (see frank-medium.container, kept
|
||||
# as the target). That shape cannot run on this host: RCCL's collective kernels
|
||||
# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
|
||||
# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
|
||||
# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
|
||||
# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
|
||||
# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
|
||||
# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
|
||||
# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
|
||||
# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
|
||||
# atomics. Result: every collective that touches GPU 2 dies at launch.
|
||||
# Two independent single-GPU replicas use both cards instead; the proxy unit
|
||||
# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
|
||||
#
|
||||
# SHAPE. vLLM binds 0.0.0.0:7771 on the host network directly; no PublishPort.
|
||||
# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
|
||||
# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
|
||||
# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
|
||||
# this shape instead of using the 2x7551-32624 pin, which is only valid at
|
||||
# TP=2/262144.
|
||||
#
|
||||
# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=0 (the first R9700). The
|
||||
# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
|
||||
# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
|
||||
# first and HIP indexes into the filtered list, so a single-valued pair
|
||||
# (ROCR=1,HIP=1) leaves an empty device set.
|
||||
#
|
||||
# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
|
||||
# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
|
||||
[Unit]
|
||||
Description=frank-medium replica A — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #1
|
||||
After=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=frank-medium-a
|
||||
Image=localhost/radiance-vllm:0.9.3
|
||||
AddDevice=/dev/kfd
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
# The upstream launcher's exact podman posture (rootless).
|
||||
PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
|
||||
Environment=HIP_VISIBLE_DEVICES=0
|
||||
Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
|
||||
Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
|
||||
Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
|
||||
Environment=SERVED_NAMES=frank-medium
|
||||
Environment=PORT=7771
|
||||
Environment=TP=1
|
||||
Environment=MAXSEQS=4
|
||||
Environment=CHUNK=8192
|
||||
Environment=MAXLEN=114688
|
||||
Environment=GPU_UTIL=0.98
|
||||
Environment=KV_MEM=0
|
||||
Environment=SPEC_METHOD=dflash
|
||||
Environment=SPEC=7
|
||||
# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
|
||||
Volume=/home/dank/models:/models
|
||||
# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
|
||||
Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
|
||||
# Compile cache. Shared with the other replica and with the TP=2 target: the
|
||||
# entries are content-addressed, so shapes simply add subdirectories.
|
||||
Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
|
||||
# The pinned radiance clone: patches applied at every container start.
|
||||
Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
|
||||
# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
|
||||
# shipped r4d.so predates the GDN overflow fix and NaNs this model.
|
||||
Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
|
||||
# Entrypoint + healthcheck from this checkout, mounted read-only.
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
|
||||
# The entrypoint must be the container command; the image's own entrypoint
|
||||
# (which the patched script execs at the end) must not run unpatched.
|
||||
PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
|
||||
# 200 only once the engine serves; during first-start compilation the
|
||||
# container is legitimately unhealthy. HealthCmd is split on spaces into an
|
||||
# argv array by the podman 5.x v1 generator, hence the file-based probe.
|
||||
HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
|
||||
HealthInterval=60s
|
||||
HealthTimeout=30s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=1800s
|
||||
Pull=never
|
||||
|
||||
[Service]
|
||||
# First start compiles Triton/inductor kernels (several minutes) before the
|
||||
# engine loads ~20 GiB of weights.
|
||||
TimeoutStartSec=1800
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,98 +0,0 @@
|
||||
# Quadlet unit: frank-medium replica B (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/frank-medium-b.container
|
||||
# Unit name: frank-medium-b.service (container name: frank-medium-a)
|
||||
#
|
||||
# WHY TWO REPLICAS INSTEAD OF TP=2. The intended shape for frank-medium is one
|
||||
# tensor-parallel process across both R9700s (see frank-medium.container, kept
|
||||
# as the target). That shape cannot run on this host: RCCL's collective kernels
|
||||
# carry a hidden hostcall-buffer parameter, and the ROCr runtime refuses to
|
||||
# dispatch them on a device without PCIe atomics ("Pcie atomics not enabled,
|
||||
# hostcall not supported" -> AQL dispatch failed -> hipErrorIllegalState). The
|
||||
# second R9700 sits behind the chipset bridges 05:00.0/06:09.0, which advertise
|
||||
# `AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
|
||||
# atomics on that path; the amdgpu driver's pci_enable_atomic_ops_to_root()
|
||||
# therefore fails for that GPU and KFD marks the CPU<->GPU2 link NO_ATOMICS.
|
||||
# The first R9700 sits on CPU root port 00:01.1 (Routing+) and does have
|
||||
# atomics. Result: every collective that touches GPU 2 dies at launch.
|
||||
# Two independent single-GPU replicas use both cards instead; the proxy unit
|
||||
# keeps the single :7770 endpoint. Full evidence: the frank-medium runbook.
|
||||
#
|
||||
# SHAPE. vLLM binds 0.0.0.0:7772 on the host network directly; no PublishPort.
|
||||
# MAXLEN is 114688, not the TP=2 deployment's 262144: one card carries the
|
||||
# whole model (~20 GiB of weights + drafter), leaving ~5.2 GiB of KV, and
|
||||
# 131072 does not fit (measured: needs 5.58 GiB). KV_MEM=0 lets vLLM profile
|
||||
# this shape instead of using the 2x7551-32624 pin, which is only valid at
|
||||
# TP=2/262144.
|
||||
#
|
||||
# DEVICES. /dev/kfd + /dev/dri; HIP_VISIBLE_DEVICES=1 (the second R9700). The
|
||||
# 2 GiB Raphael iGPU enumerates as index 2 and is excluded by the pin.
|
||||
# Only HIP_VISIBLE_DEVICES is set, never ROCR_VISIBLE_DEVICES: ROCR filters
|
||||
# first and HIP indexes into the filtered list, so a single-valued pair
|
||||
# (ROCR=1,HIP=1) leaves an empty device set.
|
||||
#
|
||||
# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
|
||||
# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
|
||||
[Unit]
|
||||
Description=frank-medium replica B — vllm-radiance MXFP4 Qwen3.8-27B on R9700 #2
|
||||
After=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=frank-medium-b
|
||||
Image=localhost/radiance-vllm:0.9.3
|
||||
AddDevice=/dev/kfd
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
# The upstream launcher's exact podman posture (rootless).
|
||||
PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
|
||||
Environment=HIP_VISIBLE_DEVICES=1
|
||||
Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
|
||||
Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
|
||||
Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
|
||||
Environment=SERVED_NAMES=frank-medium
|
||||
Environment=PORT=7772
|
||||
Environment=TP=1
|
||||
Environment=MAXSEQS=4
|
||||
Environment=CHUNK=8192
|
||||
Environment=MAXLEN=114688
|
||||
Environment=GPU_UTIL=0.98
|
||||
Environment=KV_MEM=0
|
||||
Environment=SPEC_METHOD=dflash
|
||||
Environment=SPEC=7
|
||||
# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards).
|
||||
Volume=/home/dank/models:/models
|
||||
# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the entrypoint.
|
||||
Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
|
||||
# Compile cache. Shared with the other replica and with the TP=2 target: the
|
||||
# entries are content-addressed, so shapes simply add subdirectories.
|
||||
Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
|
||||
# The pinned radiance clone: patches applied at every container start.
|
||||
Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
|
||||
# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The image's
|
||||
# shipped r4d.so predates the GDN overflow fix and NaNs this model.
|
||||
Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
|
||||
# Entrypoint + healthcheck from this checkout, mounted read-only.
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
|
||||
# The entrypoint must be the container command; the image's own entrypoint
|
||||
# (which the patched script execs at the end) must not run unpatched.
|
||||
PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
|
||||
# 200 only once the engine serves; during first-start compilation the
|
||||
# container is legitimately unhealthy. HealthCmd is split on spaces into an
|
||||
# argv array by the podman 5.x v1 generator, hence the file-based probe.
|
||||
HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
|
||||
HealthInterval=60s
|
||||
HealthTimeout=30s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=1800s
|
||||
Pull=never
|
||||
|
||||
[Service]
|
||||
# First start compiles Triton/inductor kernels (several minutes) before the
|
||||
# engine loads ~20 GiB of weights.
|
||||
TimeoutStartSec=1800
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,61 +0,0 @@
|
||||
# Quadlet unit: frank-medium proxy (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/frank-medium-proxy.container
|
||||
# Unit name: frank-medium-proxy.service (container name: frank-medium-proxy)
|
||||
#
|
||||
# WHAT IT IS. A prefix-affinity reverse proxy in front of the two single-GPU
|
||||
# frank-medium replicas (:7771 on R9700 #1, :7772 on R9700 #2). It preserves
|
||||
# the fleet contract: one endpoint on host port 7770 answering as the served
|
||||
# model name frank-medium, OpenAI /v1, streaming included.
|
||||
#
|
||||
# WHY IT EXISTS. TP=2 — one vLLM process spanning both cards — is impossible
|
||||
# on this host (see frank-medium-a.container and the runbook: RCCL's kernels
|
||||
# need PCIe atomics, and the second card's chipset path advertises
|
||||
# AtomicOpsCap: Routing-). Two replicas are how both cards get used.
|
||||
#
|
||||
# ROUTING. Requests are keyed on a hash of their leading 4 KiB of prompt text,
|
||||
# so a conversation family — an agent loop re-sending the same system prompt,
|
||||
# tool list and file context every turn — keeps landing on the same replica,
|
||||
# where --enable-prefix-caching has that prefix warm. Plain round-robin threw
|
||||
# half of those cache hits away. A replica that is down is skipped, and
|
||||
# ROUTING=roundrobin restores the old behaviour for A/B measurement. /health
|
||||
# answers 200 while at least one replica is up and names the ones that are
|
||||
# down. The proxy runs on host networking so it can bind 7770 and reach the
|
||||
# replicas on loopback.
|
||||
#
|
||||
# The script is a few dozen lines of aiohttp, mounted read-only out of
|
||||
# /home/dank/serverAppConfigs (deploy.sh pulls that checkout before dispatch).
|
||||
# It reuses the same image as the replicas: aiohttp is already there and no
|
||||
# extra image has to be pulled on this host.
|
||||
[Unit]
|
||||
Description=frank-medium proxy — prefix-affinity in front of the two single-GPU replicas
|
||||
After=network-online.target
|
||||
Wants=frank-medium-a.service frank-medium-b.service
|
||||
After=frank-medium-a.service frank-medium-b.service
|
||||
|
||||
[Container]
|
||||
ContainerName=frank-medium-proxy
|
||||
Image=localhost/radiance-vllm:0.9.3
|
||||
PodmanArgs=--network=host
|
||||
Environment=PORT=7770
|
||||
Environment=UPSTREAMS=http://127.0.0.1:7771,http://127.0.0.1:7772
|
||||
# The proxy is two files: routing.py holds the replica-selection policy, the
|
||||
# server holds the HTTP plumbing. Both are mounted because the server imports
|
||||
# the policy by name from its own directory (/usr/local/bin is sys.path[0]).
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/routing.py:/usr/local/bin/routing.py:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/proxy/affinity-proxy.py:/usr/local/bin/affinity-proxy.py:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
|
||||
PodmanArgs=--entrypoint /usr/local/bin/affinity-proxy.py
|
||||
HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
|
||||
HealthInterval=60s
|
||||
HealthTimeout=30s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=60s
|
||||
Pull=never
|
||||
|
||||
[Service]
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,129 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Routing policy for the frank-medium proxy: which replica serves a request.
|
||||
|
||||
Kept apart from the HTTP plumbing in affinity-proxy.py so the decisions can be
|
||||
tested without aiohttp, a network or a GPU — the first version of these rules
|
||||
shipped with a bug that only appeared on a continuation request, and a test
|
||||
that loads this module catches that class of mistake directly.
|
||||
|
||||
Two cards, two behaviours, and the workload decides which is right:
|
||||
|
||||
* A **single-turn** request is one member of a batch (a benchmark run, a burst
|
||||
of parallel subagents). Spreading those over both replicas lets both cards
|
||||
prefill at once; measured on this host, a 16-request burst of 15k-token
|
||||
prompts finishes in 18.6 s spread versus 30.1 s pinned.
|
||||
* A **continuation** re-sends a prefix the replica has already prefilled. Pinned
|
||||
to that replica the turn costs a cache lookup; spread, half the turns pay a
|
||||
cold prefill again (measured: 1096 ms versus 2676 ms mean warm-turn TTFT on
|
||||
90k-token contexts).
|
||||
|
||||
ROUTING picks the strategy; the default `adaptive` applies both rules above.
|
||||
`affinity` and `roundrobin` exist so the choice can be re-measured, and are not
|
||||
production settings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import collections
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
|
||||
UPSTREAMS = [u.strip() for u in os.environ.get(
|
||||
"UPSTREAMS", "http://127.0.0.1:7771,http://127.0.0.1:7772").split(",") if u.strip()]
|
||||
ROUTING = os.environ.get("ROUTING", "adaptive").strip().lower()
|
||||
# Leading bytes of prompt text that identify a conversation's shared prefix.
|
||||
# 4 KiB covers a system prompt plus the start of the first user turn, which is
|
||||
# where agent loops keep the text they resend every turn.
|
||||
AFFINITY_BYTES = int(os.environ.get("AFFINITY_BYTES", "4096"))
|
||||
# Prefixes remembered for pinning. Bounded so a long-lived proxy cannot grow
|
||||
# without limit; 4096 conversations is far beyond what this fleet runs at once.
|
||||
KNOWN_MAX = int(os.environ.get("KNOWN_MAX", "4096"))
|
||||
|
||||
_counter = itertools.count()
|
||||
# digest -> replica index, insertion-ordered for O(1) LRU eviction. A prefix is
|
||||
# remembered as soon as any request carries it; only a continuation is pinned
|
||||
# there, because only a continuation can profit from the cache entry.
|
||||
_known: "collections.OrderedDict[bytes, int]" = collections.OrderedDict()
|
||||
|
||||
|
||||
def affinity_key(body: bytes, remote: str | None) -> tuple[bytes, bool]:
|
||||
"""Key a request by its shared prefix, and report whether it continues one.
|
||||
|
||||
Chat and completions bodies both carry their prompt text in JSON, so the
|
||||
leading bytes of that text are the prefix; a body we cannot parse (or an
|
||||
endpoint with no prompt, e.g. /v1/models) falls back to the client address,
|
||||
which still beats nothing for a single-client loop.
|
||||
"""
|
||||
fallback = (remote or "anonymous").encode()
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except Exception:
|
||||
return fallback, False
|
||||
|
||||
text = ""
|
||||
continuation = False
|
||||
if isinstance(payload, dict):
|
||||
if isinstance(payload.get("prompt"), str):
|
||||
text = payload["prompt"]
|
||||
elif isinstance(payload.get("messages"), list):
|
||||
continuation = len(payload["messages"]) > 1
|
||||
parts = []
|
||||
for msg in payload["messages"]:
|
||||
if isinstance(msg, dict):
|
||||
content = msg.get("content")
|
||||
if isinstance(content, str):
|
||||
parts.append(content)
|
||||
elif isinstance(content, list): # multimodal content blocks
|
||||
parts.extend(
|
||||
blk.get("text", "") for blk in content
|
||||
if isinstance(blk, dict) and isinstance(blk.get("text"), str)
|
||||
)
|
||||
text = "\n".join(parts)
|
||||
if not text:
|
||||
return fallback, continuation
|
||||
return text[:AFFINITY_BYTES].encode("utf-8", "replace"), continuation
|
||||
|
||||
|
||||
def pick(key: bytes, continuation: bool, up: list[bool]) -> int:
|
||||
"""Replica for a request, or -1 when none is available.
|
||||
|
||||
Down replicas are never selected — a rollout must not send traffic to an
|
||||
engine that is still compiling its graphs — and a pin whose replica is down
|
||||
is re-established on a live one rather than kept pointing at a corpse.
|
||||
"""
|
||||
digest = hashlib.sha1(key).digest()
|
||||
|
||||
if ROUTING == "roundrobin":
|
||||
start = next(_counter) % len(UPSTREAMS)
|
||||
elif ROUTING == "adaptive":
|
||||
entry = _known.get(digest)
|
||||
if continuation and entry is not None and up[entry]:
|
||||
_known.move_to_end(digest)
|
||||
return entry
|
||||
# Anything that is not a continuation spreads by counter — including a
|
||||
# batch of requests that happen to share one prefix. Hashing the prefix
|
||||
# would send every member of such a batch to whichever card the hash
|
||||
# names (measured: 29.8 s for a 16-request burst, against 18.6 s when
|
||||
# spread). The counter is only advanced by these requests, so batch
|
||||
# members alternate across the cards.
|
||||
start = next(_counter) % len(UPSTREAMS)
|
||||
else: # affinity
|
||||
start = int.from_bytes(digest[:8], "big") % len(UPSTREAMS)
|
||||
|
||||
for offset in range(len(UPSTREAMS)):
|
||||
idx = (start + offset) % len(UPSTREAMS)
|
||||
if up[idx]:
|
||||
if ROUTING == "adaptive":
|
||||
_known[digest] = idx
|
||||
_known.move_to_end(digest)
|
||||
if len(_known) > KNOWN_MAX:
|
||||
_known.popitem(last=False)
|
||||
return idx
|
||||
return -1
|
||||
|
||||
|
||||
def route(body: bytes, remote: str | None, up: list[bool]) -> int:
|
||||
"""Convenience wrapper: decode the request body and choose a replica."""
|
||||
return pick(*affinity_key(body, remote), up)
|
||||
@@ -1,22 +0,0 @@
|
||||
# Dankserver stack secrets and configuration.
|
||||
# Copy to .env and fill real values. Never commit real secrets.
|
||||
|
||||
# RomM admin credentials (first boot)
|
||||
ROMM_USERNAME=changeme
|
||||
ROMM_PASSWORD=changeme
|
||||
ROMM_AUTH_SECRET_KEY=changeme
|
||||
|
||||
# RomM database (MariaDB)
|
||||
ROMM_DB_USER=romm
|
||||
ROMM_DB_PASSWD=changeme
|
||||
MARIADB_ROOT_PASSWORD=changeme
|
||||
|
||||
# RomM Redis
|
||||
# (no password needed for local Redis)
|
||||
|
||||
# Copyparty
|
||||
COPAY_PASS=changeme
|
||||
|
||||
# Jellyfin
|
||||
JELLYFIN_PUBLISH_SERVER_URL=http://192.168.0.235:8096
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
# Quadlet unit: Calibre Content Server (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver:
|
||||
# ~/.config/containers/systemd/calibre-server.container
|
||||
# Unit name: calibre-server.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The image is the CURRENT linuxserver/calibre (9.15.0), which is a desktop
|
||||
# image (Selkies web-streamed GUI on 8080/8181) with the headless
|
||||
# calibre-server available inside. The compose-era snippet in
|
||||
# serverAppConfigs-docs/Dankserver/Calibre.md predates the desktop revamp;
|
||||
# the dedicated headless images (ta264/docker-calibre, agentgreen) are
|
||||
# dormant (calibre 6.10 / 2017), so this unit runs calibre-server as the
|
||||
# image's s6-overlay main service via Exec=; the desktop also runs but its
|
||||
# ports are internal-only (see the port map below).
|
||||
#
|
||||
# Port map: Network=pasta is port-FORWARD mode on this host (the container
|
||||
# keeps its own netns; only PublishPort'd ports reach the host — verified
|
||||
# 2026-09-21: 8080/8090/8182 inside the container are not visible on the
|
||||
# host, only the 8084 forward). So the ONLY exposed surface is:
|
||||
# 8090 -> host 8084 calibre-server (the content server; 8080 inside the
|
||||
# container is the image's own nginx, 8083 on the host
|
||||
# is calibre-web)
|
||||
# The desktop GUI (nginx 8080/8181 + WS 8082, inside the container) runs but
|
||||
# is internal-only: no PublishPort. Exposing it is one PublishPort line away
|
||||
# if the browser GUI is ever wanted (the basic-auth credential is already in
|
||||
# the env file).
|
||||
# Auth: calibre-server 9.x keeps its users in a SQLite userdb (the pw column
|
||||
# is PLAINTEXT, so the file is mode 600). The single admin account is
|
||||
# bootstrapped once by the deploy handler (deploy_calibre_server) via
|
||||
# `calibre-server --manage-users -- add`, password piped on stdin with
|
||||
# printf (a trailing newline from echo lands in the stored password and
|
||||
# breaks every login). The desktop's basic auth (CUSTOM_USER/CUSTOM_PASSWORD)
|
||||
# reuses the same credential pair. Both live in the machine-local env file;
|
||||
# this unit holds no secrets.
|
||||
#
|
||||
# --auth-mode=basic is required: on a non-SSL port the server defaults to
|
||||
# Digest auth, which rejects plain Basic clients with 400 "Unsupported
|
||||
# authentication method".
|
||||
|
||||
[Unit]
|
||||
Description=Calibre content server container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=calibre-server
|
||||
Image=lscr.io/linuxserver/calibre:9.15.0
|
||||
Network=pasta
|
||||
EnvironmentFile=/home/dank/Containers/calibre-server/env
|
||||
Environment=PUID=1000
|
||||
Environment=PGID=1000
|
||||
Environment=TZ=America/Vancouver
|
||||
Volume=/home/dank/Containers/calibre-web/books:/books
|
||||
Volume=/home/dank/Containers/calibre-server/userdb.sqlite:/userdb.sqlite
|
||||
Exec=calibre-server --port 8090 --enable-auth --auth-mode=basic --userdb /userdb.sqlite /books
|
||||
PublishPort=8084:8090
|
||||
|
||||
[Service]
|
||||
TimeoutStartSec=300
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,43 +0,0 @@
|
||||
# Quadlet unit: Calibre-Web library server (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver:
|
||||
# ~/.config/containers/systemd/calibre-web.container
|
||||
# Unit name: calibre-web.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The data roots already lived under /home/dank/Containers/calibre-web in the
|
||||
# compose era; this unit keeps those exact paths. The image pin is the exact
|
||||
# tag of the image the live container runs (image id 986277563e76), so the
|
||||
# cutover recreation changes zero bits of application code. No secrets: this
|
||||
# unit takes no EnvironmentFile.
|
||||
|
||||
[Unit]
|
||||
Description=Calibre-Web library server container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=calibre-web
|
||||
Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402
|
||||
Network=pasta
|
||||
Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
|
||||
Environment=LANG=en_US.UTF-8
|
||||
Environment=LANGUAGE=en_US.UTF-8
|
||||
Environment=LSIO_FIRST_PARTY=true
|
||||
Environment=OAUTHLIB_RELAX_TOKEN_SCOPE=1
|
||||
Environment=PGID=1000
|
||||
Environment=PUID=1000
|
||||
Environment=QTWEBENGINE_CHROMIUM_FLAGS=--no-sandbox
|
||||
Environment=S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
|
||||
Environment=S6_STAGE2_HOOK=/docker-mods
|
||||
Environment=TERM=xterm
|
||||
Environment=TZ=America/Vancouver
|
||||
Environment=VIRTUAL_ENV=/lsiopy
|
||||
PublishPort=8083:8083
|
||||
Volume=/home/dank/Containers/calibre-web/config:/config
|
||||
Volume=/home/dank/Containers/calibre-web/books:/books
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,154 +0,0 @@
|
||||
# Dankserver stack — media & library services.
|
||||
# Deploy: cd /home/dank/serverAppConfigs && podman-compose -f dankserver/compose.yaml up -d
|
||||
# See ../docs/Dankserver/ for service documentation.
|
||||
|
||||
networks:
|
||||
appnet:
|
||||
driver: bridge
|
||||
|
||||
services:
|
||||
# ═══════════════════════════════════════════════
|
||||
# RomM — retro game manager (requires MariaDB + Redis)
|
||||
# ═══════════════════════════════════════════════
|
||||
mariadb:
|
||||
image: docker.io/mariadb:11
|
||||
container_name: romm-mariadb
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MARIADB_ROOT_PASSWORD: "${MARIADB_ROOT_PASSWORD}"
|
||||
MARIADB_DATABASE: romm
|
||||
MARIADB_USER: "${ROMM_DB_USER}"
|
||||
MARIADB_PASSWORD: "${ROMM_DB_PASSWD}"
|
||||
volumes:
|
||||
- /opt/romm/mariadb:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "mariadb-admin", "ping", "-h", "localhost"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
|
||||
redis:
|
||||
image: docker.io/redis:7-alpine
|
||||
container_name: romm-redis
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /opt/romm/redis:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
|
||||
romm:
|
||||
image: ghcr.io/rommapp/romm:latest
|
||||
container_name: romm
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
ROMM_USERNAME: "${ROMM_USERNAME}"
|
||||
ROMM_PASSWORD: "${ROMM_PASSWORD}"
|
||||
ROMM_AUTH_SECRET_KEY: "${ROMM_AUTH_SECRET_KEY}"
|
||||
DB_HOST: mariadb
|
||||
DB_NAME: romm
|
||||
DB_USER: "${ROMM_DB_USER}"
|
||||
DB_PASSWD: "${ROMM_DB_PASSWD}"
|
||||
REDIS_HOST: redis
|
||||
HASHEOUS_API_ENABLED: "true"
|
||||
ports:
|
||||
- "8085:8080"
|
||||
volumes:
|
||||
- /opt/romm/library:/romm/library
|
||||
- /opt/romm/resources:/romm/resources
|
||||
- /opt/romm/config:/romm/config
|
||||
depends_on:
|
||||
mariadb:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Copyparty — file sharing and streaming
|
||||
# ═══════════════════════════════════════════════
|
||||
copyparty:
|
||||
image: ghcr.io/9001/copyparty-ac:latest
|
||||
container_name: copyparty
|
||||
restart: unless-stopped
|
||||
# read-only since 2026-08-31: copyparty perms are per-field "letters[,user]"
|
||||
# with no user = every unauthenticated request. Anonymous gets r (browse +
|
||||
# download; the guarded files.franksdank.cloud chain reaches copyparty
|
||||
# unauthenticated after authentik). dank gets ra (admin panel). No w/m/d
|
||||
# anywhere: uploads, moves, deletes rejected server-side. Interface: `--help-accounts` in-image.
|
||||
command: ["-a", "dank:${COPAY_PASS}", "-v", "/w::r:ra,dank"]
|
||||
ports:
|
||||
- "3923:3923"
|
||||
volumes:
|
||||
- /mnt/entropy:/w:ro
|
||||
- /opt/copyparty/cfg:/cfg
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:3923/')"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Jellyfin — media server (movies, TV, music)
|
||||
# ═══════════════════════════════════════════════
|
||||
jellyfin:
|
||||
image: jellyfin/jellyfin:latest
|
||||
container_name: jellyfin
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8096:8096/tcp"
|
||||
- "8920:8920/tcp"
|
||||
- "7359:7359/udp"
|
||||
- "1900:1900/udp"
|
||||
environment:
|
||||
- JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
|
||||
- DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
|
||||
volumes:
|
||||
- /opt/jellyfin/config:/config
|
||||
- /opt/jellyfin/cache:/cache
|
||||
- /mnt/entropy:/data:ro
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
group_add:
|
||||
- "992"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Calibre-Web — e-book web frontend
|
||||
# ═══════════════════════════════════════════════
|
||||
calibre-web:
|
||||
image: lscr.io/linuxserver/calibre-web:0.6.27-ls402
|
||||
container_name: calibre-web
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
PUID: "1000"
|
||||
PGID: "1000"
|
||||
TZ: Etc/UTC
|
||||
OAUTHLIB_RELAX_TOKEN_SCOPE: "1"
|
||||
ports:
|
||||
- "8083:8083"
|
||||
volumes:
|
||||
- /opt/calibre-web/config:/config
|
||||
- /opt/calibre-web/books:/books
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8083/"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
|
||||
# node-exporter left this file on 2026-09-16 for
|
||||
# dankserver/observability/quadlet/node-exporter.container: a compose-only
|
||||
# container does not survive a reboot here — it died `Exited (143)` and stayed
|
||||
# dead, leaving Prometheus' node-dankserver job scraping nothing — and the
|
||||
# compose form bound the wildcard address. One owner per container name.
|
||||
@@ -1,38 +0,0 @@
|
||||
# Quadlet unit: Copyparty file server (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/copyparty.container
|
||||
# Unit name: copyparty.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The image's default entrypoint is /bin/ash /z/cpp.sh -c /z/initcfg; the
|
||||
# Command= line below reproduces the compose-era argv exactly: the admin
|
||||
# account is passed as -a <user>:<password>. ${COPAY_ACCOUNT} and
|
||||
# ${COPAY_PASSWORD} are expanded by systemd at ExecStart time from the
|
||||
# host-local env file /home/dank/Containers/copyparty/env (mode 0600).
|
||||
# This file holds no secrets. The deployer refuses to start this unit before
|
||||
# that env file exists, so a missing env file fails the unit instead of
|
||||
# creating an admin account with a garbage password.
|
||||
#
|
||||
# /w is the entropy volume, mounted read-only: nothing writes into /mnt/entropy
|
||||
# anymore; every consumer of the share is a read-only client.
|
||||
|
||||
[Unit]
|
||||
Description=Copyparty file server container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=copyparty
|
||||
Image=ghcr.io/9001/copyparty-ac:latest
|
||||
Network=dankserver_appnet
|
||||
EnvironmentFile=/home/dank/Containers/copyparty/env
|
||||
Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD} -v /w::r:ra,dank
|
||||
Volume=/mnt/entropy:/w:ro
|
||||
Volume=/home/dank/Containers/copyparty/cfg:/cfg
|
||||
PublishPort=3923:3923
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/copyparty/env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,57 +0,0 @@
|
||||
# Quadlet unit: Jellyfin media server (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/jellyfin.container
|
||||
# Unit name: jellyfin.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
|
||||
# cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
|
||||
# anywhere writes to the entropy volume. Every
|
||||
# creation-time environment variable of the compose-era container is kept as
|
||||
# an Environment= line (zero drift); none of them are secrets, so this unit
|
||||
# needs no EnvironmentFile.
|
||||
|
||||
[Unit]
|
||||
Description=Jellyfin media server container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=jellyfin
|
||||
Image=ghcr.io/jellyfin/jellyfin:latest
|
||||
Network=dankserver_appnet
|
||||
Environment=DEBIAN_FRONTEND=noninteractive
|
||||
Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
|
||||
Environment=JELLYFIN_CACHE_DIR=/cache
|
||||
Environment=JELLYFIN_CONFIG_DIR=/config/config
|
||||
Environment=JELLYFIN_DATA_DIR=/config
|
||||
Environment=JELLYFIN_FFMPEG=/usr/lib/jellyfin-ffmpeg/ffmpeg
|
||||
Environment=JELLYFIN_LOG_DIR=/config/log
|
||||
Environment=JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
|
||||
Environment=JELLYFIN_WEB_DIR=/jellyfin/jellyfin-web
|
||||
Environment=LANG=en_US.UTF-8
|
||||
Environment=LANGUAGE=en_US:en
|
||||
Environment=LC_ALL=en_US.UTF-8
|
||||
Environment=LD_PRELOAD=/usr/lib/jellyfin/libjemalloc.so.2
|
||||
Environment=MALLOC_TRIM_THRESHOLD_=131072
|
||||
Environment=NVIDIA_DRIVER_CAPABILITIES=compute,video,utility
|
||||
Environment=NVIDIA_VISIBLE_DEVICES=all
|
||||
Environment=XDG_CACHE_HOME=/cache
|
||||
PublishPort=8096:8096
|
||||
PublishPort=8920:8920
|
||||
PublishPort=7359:7359/udp
|
||||
PublishPort=1900:1900/udp
|
||||
Volume=/home/dank/Containers/jellyfin/config:/config
|
||||
Volume=/home/dank/Containers/jellyfin/cache:/cache
|
||||
Volume=/mnt/entropy:/data:ro
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
HealthCmd=curl -f http://localhost:8096/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,39 +0,0 @@
|
||||
# Quadlet unit: cadvisor (rootless, boot-persistent) — DANKSERVER.
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/cadvisor.container
|
||||
# Unit name: cadvisor.service (container name: cadvisor)
|
||||
#
|
||||
# The Dankserver twin of dankclaw/observability/quadlet/cadvisor.container.
|
||||
# Same rootless shape, same --cgroupns=host requirement (see that file for the
|
||||
# full story), only the LAN address differs: this host publishes on
|
||||
# 192.168.0.235:8080 so Prometheus on Dankclaw can scrape it over the real
|
||||
# network (cross-host, so the host.containers.internal trick does not apply).
|
||||
# It reports the media/library stack (romm, plex, jellyfin, copyparty, calibre,
|
||||
# tautulli) by their quadlet unit names via the cgroup path.
|
||||
|
||||
[Unit]
|
||||
Description=cAdvisor container metrics (Dankserver)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=cadvisor
|
||||
Image=gcr.io/cadvisor/cadvisor:v0.49.1
|
||||
PodmanArgs=--pid=host --cgroupns=host
|
||||
PublishPort=127.0.0.1:8080:8080
|
||||
PublishPort=192.168.0.235:8080:8080
|
||||
Volume=/sys:/sys:ro
|
||||
Volume=/:/rootfs:ro
|
||||
Exec=--housekeeping_interval=30s
|
||||
HealthCmd=wget --spider -q http://localhost:8080/healthz
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=40s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,58 +0,0 @@
|
||||
# Quadlet unit: node-exporter (rootless, boot-persistent) — DANKSERVER.
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
|
||||
# Unit name: node-exporter.service (container name: node-exporter)
|
||||
#
|
||||
# WHY THIS FILE EXISTS. Until 2026-09-16 the Dankserver node-exporter ran only
|
||||
# from `dankserver/compose.yaml` (the "Observability" block, `restart:
|
||||
# unless-stopped`). Rootless Podman does not restart bare containers after the
|
||||
# explicit stop that shutdown performs, so it died at a reboot and stayed dead:
|
||||
# `podman ps -a` showed it `Exited (143)` and Prometheus' `node-dankserver` job
|
||||
# had been scraping nothing since. The compose entry also bound the default
|
||||
# wildcard address, which is the unauthenticated-metrics-API-on-every-interface
|
||||
# posture that was already removed on Dankclaw. This unit replaces both: one
|
||||
# owner per container name (the compose service is deleted), explicit listen
|
||||
# addresses, and a generator-provided boot path.
|
||||
#
|
||||
# HOST NAMESPACE. Networking and PID in the host's namespaces: the collectors
|
||||
# report the host's own interfaces, connections, processes and cgroups. The
|
||||
# host's /proc, /sys and / are mounted read-only under /host and /rootfs, and
|
||||
# node_exporter writes nothing.
|
||||
#
|
||||
# LISTEN ADDRESSES. Exactly two:
|
||||
# - 127.0.0.1:9100 for host-local probes
|
||||
# - 192.168.0.235:9100 for Prometheus, which scrapes this host from Dankclaw
|
||||
# (shared/prometheus/prometheus.yml, job "node-dankserver"). It cannot use
|
||||
# loopback across hosts, and node_exporter has no authentication, so the
|
||||
# wildcard bind stays off the table. This is the same deliberate LAN
|
||||
# listener posture the observability runbook documents.
|
||||
#
|
||||
# MIRROR. Keep this file in step with dankclaw/observability/quadlet/
|
||||
# node-exporter.container; only the Description and the LAN address differ.
|
||||
|
||||
[Unit]
|
||||
Description=Prometheus node exporter (Dankserver host metrics)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=node-exporter
|
||||
Image=docker.io/prom/node-exporter:latest
|
||||
Network=host
|
||||
PodmanArgs=--pid=host
|
||||
Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:9100 --web.listen-address=192.168.0.235:9100
|
||||
Volume=/proc:/host/proc:ro
|
||||
Volume=/sys:/host/sys:ro
|
||||
Volume=/:/rootfs:ro
|
||||
HealthCmd=wget --spider -q http://localhost:9100/metrics
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
HealthStartPeriod=30s
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,22 +0,0 @@
|
||||
[Unit]
|
||||
Description=Plex Media Server (quadlet, rootless fleet identity)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
Image=lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329
|
||||
ContainerName=plex
|
||||
Network=dankserver_appnet:alias=plex
|
||||
Environment=PUID=0
|
||||
Environment=PGID=0
|
||||
Environment=TZ=America/Los_Angeles
|
||||
Volume=/var/lib/plexmediaserver:/config
|
||||
Volume=/mnt/entropy:/mnt/entropy:ro
|
||||
PublishPort=32400:32400
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,34 +0,0 @@
|
||||
# Quadlet unit: MariaDB backing store for RomM (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/romm-mariadb.container
|
||||
# Unit name: romm-mariadb.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The alias `mariadb` on dankserver_appnet is what makes DB_HOST=mariadb in
|
||||
# the RomM unit resolve. Secrets (MARIADB_ROOT_PASSWORD, MARIADB_PASSWORD)
|
||||
# live in the shared host-local env file /home/dank/Containers/romm/env
|
||||
# (mode 0600); this file holds no secrets.
|
||||
|
||||
[Unit]
|
||||
Description=RomM MariaDB database container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm-mariadb
|
||||
Image=docker.io/library/mariadb:11
|
||||
Network=dankserver_appnet:alias=mariadb
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Environment=MARIADB_DATABASE=romm
|
||||
Environment=MARIADB_USER=romm
|
||||
Volume=/home/dank/Containers/romm/mariadb:/var/lib/mysql
|
||||
HealthCmd=mariadb-admin ping -h localhost
|
||||
HealthInterval=10s
|
||||
HealthTimeout=5s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,28 +0,0 @@
|
||||
# Quadlet unit: Redis cache for RomM (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/romm-redis.container
|
||||
# Unit name: romm-redis.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The alias `redis` on dankserver_appnet is what makes REDIS_HOST=redis in
|
||||
# the RomM unit resolve. No secrets: this unit takes no EnvironmentFile.
|
||||
|
||||
[Unit]
|
||||
Description=RomM Redis cache container
|
||||
After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm-redis
|
||||
Image=docker.io/library/redis:7-alpine
|
||||
Network=dankserver_appnet:alias=redis
|
||||
Volume=/home/dank/Containers/romm/redis:/data
|
||||
HealthCmd=redis-cli ping
|
||||
HealthInterval=10s
|
||||
HealthTimeout=5s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,42 +0,0 @@
|
||||
# Quadlet unit: RomM library manager (rootless, boot-persistent).
|
||||
#
|
||||
# Install path on dankserver: ~/.config/containers/systemd/romm.container
|
||||
# Unit name: romm.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# The whole data root lives at /home/dank/Containers/romm and is mounted at
|
||||
# /romm, so config/, library/, resources/ and assets/ sit inside it exactly as
|
||||
# the compose-era bind mounts + anonymous volume did. Secrets (DB_PASSWD,
|
||||
# ROMM_AUTH_SECRET_KEY, ROMM_PASSWORD, SCREENSCRAPER_DEV_*) live in the
|
||||
# host-local env file /home/dank/Containers/romm/env (mode 0600); this file
|
||||
# holds no secrets. The network dankserver_appnet must exist before first
|
||||
# start (it is the compose-era network; on a fresh host create it first).
|
||||
|
||||
[Unit]
|
||||
Description=RomM library manager container
|
||||
Wants=romm-mariadb.service
|
||||
After=romm-mariadb.service romm-redis.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm
|
||||
Image=ghcr.io/rommapp/romm:latest
|
||||
Network=dankserver_appnet
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Environment=DB_HOST=mariadb
|
||||
Environment=DB_NAME=romm
|
||||
Environment=DB_USER=romm
|
||||
Environment=REDIS_HOST=redis
|
||||
Environment=HASHEOUS_API_ENABLED=true
|
||||
PublishPort=8085:8080
|
||||
Volume=/home/dank/Containers/romm:/romm
|
||||
HealthCmd=curl -f http://localhost:8080/health
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -1,25 +0,0 @@
|
||||
[Unit]
|
||||
Description=Tautulli (quadlet, rootless fleet identity)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245
|
||||
ContainerName=tautulli
|
||||
Network=dankserver_appnet:alias=tautulli
|
||||
Environment=PUID=0
|
||||
Environment=PGID=0
|
||||
Environment=TZ=America/Los_Angeles
|
||||
Volume=/home/dank/Containers/tautulli:/config
|
||||
PublishPort=8181:8181
|
||||
HealthCmd=curl -fs http://localhost:8181/home
|
||||
HealthInterval=30s
|
||||
HealthTimeout=10s
|
||||
HealthRetries=3
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# deploy.sh — Deploy inference scripts to remote servers
|
||||
#
|
||||
# Usage: ./deploy.sh [TARGET] [SCRIPT]
|
||||
# TARGET: dankserver, dankclaw, or all
|
||||
# SCRIPT: Script name to deploy (default: all scripts in scripts/)
|
||||
|
||||
TARGET="${1:-all}"
|
||||
SCRIPT="${2:-}"
|
||||
|
||||
# Remote configuration
|
||||
declare -A REMOTE_HOSTS
|
||||
REMOTE_HOSTS=(
|
||||
["dankserver"]="dank@dankserver"
|
||||
["dankclaw"]="dank@dankclaw"
|
||||
)
|
||||
|
||||
declare -A REMOTE_DIRS
|
||||
REMOTE_DIRS=(
|
||||
["dankserver"]="/home/dank/vllm"
|
||||
["dankclaw"]="/home/dank/vllm"
|
||||
)
|
||||
|
||||
declare -A REMOTE_SERVICES
|
||||
REMOTE_SERVICES=(
|
||||
["dankserver"]="vllm-awq-qwen36.service"
|
||||
["dankclaw"]="vllm-thinkingcap.service"
|
||||
)
|
||||
|
||||
deploy_to() {
|
||||
local target="$1"
|
||||
local remote="${REMOTE_HOSTS[$target]}"
|
||||
local remote_dir="${REMOTE_DIRS[$target]}"
|
||||
local service="${REMOTE_SERVICES[$target]}"
|
||||
|
||||
echo "=== Deploying to $target ==="
|
||||
|
||||
# Create remote directory if it doesn't exist
|
||||
sshpass -p 'Assface.3241' ssh "$remote" "mkdir -p $remote_dir"
|
||||
|
||||
# Deploy scripts
|
||||
if [ -n "$SCRIPT" ]; then
|
||||
if [ -f "scripts/$SCRIPT" ]; then
|
||||
echo " Deploying scripts/$SCRIPT..."
|
||||
sshpass -p 'Assface.3241' scp "scripts/$SCRIPT" "$remote:$remote_dir/$SCRIPT"
|
||||
else
|
||||
echo " ERROR: scripts/$SCRIPT not found"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
for script in scripts/*.sh; do
|
||||
if [ -f "$script" ]; then
|
||||
local name=$(basename "$script")
|
||||
echo " Deploying $name..."
|
||||
sshpass -p 'Assface.3241' scp "$script" "$remote:$remote_dir/$name"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Make scripts executable
|
||||
if [ -n "$SCRIPT" ]; then
|
||||
sshpass -p 'Assface.3241' ssh "$remote" "chmod +x $remote_dir/$SCRIPT"
|
||||
else
|
||||
sshpass -p 'Assface.3241' ssh "$remote" "chmod +x $remote_dir/*.sh"
|
||||
fi
|
||||
|
||||
# Restart service if it exists
|
||||
echo " Restarting $service..."
|
||||
sshpass -p 'Assface.3241' ssh "$remote" "echo 'Assface.3241' | sudo -S systemctl restart $service 2>/dev/null || echo 'Service $service not found or failed'"
|
||||
|
||||
echo " Deploy to $target complete"
|
||||
echo ""
|
||||
}
|
||||
|
||||
case "$TARGET" in
|
||||
dankserver|dankclaw)
|
||||
deploy_to "$TARGET"
|
||||
;;
|
||||
all)
|
||||
for target in "${!REMOTE_HOSTS[@]}"; do
|
||||
deploy_to "$target"
|
||||
done
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Unknown target $TARGET"
|
||||
echo "Usage: $0 [dankserver|dankclaw|all] [script]"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "=== Deployment complete ==="
|
||||
@@ -0,0 +1,81 @@
|
||||
# Dankclaw System Specs
|
||||
|
||||
## Server Overview
|
||||
|
||||
Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
|
||||
## Hardware Specifications
|
||||
|
||||
### CPU
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
| Architecture | x86_64 |
|
||||
| Cores | 16 (1 socket, 1 thread per core) |
|
||||
| Max MHz | 5187.50 |
|
||||
| Min MHz | 625.00 |
|
||||
| NUMA Nodes | 1 (node 0, 62909 MB) |
|
||||
|
||||
### GPU (Integrated)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Model | AMD Radeon 8060S (Strix Halo) |
|
||||
| Device ID | 0x1586 |
|
||||
| Architecture | gfx1151 (Strix Halo) |
|
||||
| GPU-Visible Memory | 96 GiB (ROCm addressable range) |
|
||||
| Physical RAM | 30 GiB (shared CPU + GPU) |
|
||||
|
||||
### ROCm
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| ROCk Module | Loaded |
|
||||
| Runtime Version | 1.18 |
|
||||
| ROCm Version | 7.2.4 (from AMD repos) |
|
||||
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
| GPU Arch | gfx1151 (Strix Halo) |
|
||||
| GPU Global Memory | 96 GiB (overcommitted, physical is 30 GiB) |
|
||||
|
||||
### Memory
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| System RAM | 30 GiB (shared CPU + GPU) |
|
||||
| GPU-Visible (ROCm) | 96 GiB (overcommitted addressable range) |
|
||||
| Swap | 8.0 GiB |
|
||||
| /dev/shm | ~15 GiB (half of RAM) |
|
||||
|
||||
### Storage
|
||||
|
||||
| Filesystem | Size | Used | Avail | Use% |
|
||||
|------------|------|------|-------|------|
|
||||
| / (nvme0n1p2) | 1.9T | 391G | 1.4T | 22% |
|
||||
|
||||
### OS and Software
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
||||
| Kernel | 7.0.0-29-generic |
|
||||
| Python | 3.14.4 (system), 3.12.13 (vLLM venv) |
|
||||
| vLLM | 0.26.0+rocm723 |
|
||||
| uv | 0.12.2 |
|
||||
|
||||
## VLLM Relevance
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- ROCm reports 96 GiB GPU memory (overcommitted). Physical RAM is 30 GiB. vLLM sees 96 GiB and will try to use it. Set `--gpu-memory-utilization` conservatively (0.80-0.85) to avoid OOM when the OS needs memory.
|
||||
- Only 8 GiB swap. Set `--swap-space` conservatively (4-8 GiB).
|
||||
- ROCm 7.2.4 from AMD repos. vLLM ROCm support requires ROCm 6.3+.
|
||||
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Use a Python 3.12 venv.
|
||||
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
|
||||
- 1.4 TB free disk space is ample for model storage.
|
||||
- `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[VLLM Overview]]
|
||||
- [[Shared Memory Architecture Dankclaw]]
|
||||
- [[RAM CPU Offloading in VLLM]]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user