Frank
c2116658f8
The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:
arm I llama.cpp layer split 32:33 + DFlash2, Q8_0 geo decode 45.7 t/s, TTFT 0.72 s, burst-4 71.7 t/s
arm L llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4 54.3 t/s
arm R radiance vLLM TP=2 + DFlash2-FP8, MXFP4 geo decode 95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s
Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.
- un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
The replica pair and its affinity proxy stay archived; one process owns
both cards now.
- retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
stay: they are the tooling that rollback needs.
- deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
the drafter, the pinned patch clone and the patched libr4d; retires the
llama root unit into the home trash instead of relinking it; then
install_quadlet + wait_container_healthy 1800s (first start compiles).
retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
a/b/proxy trio only, since frank-medium is installed again.
- planner + test-changed map dankserver/frank-medium/** to frank-medium
and its seam; the scripts-trio mapping is gone with the files.
- tests/frank_medium_unit.bats rewritten for the container contract: boot
persistence of a GENERATED unit (no enable on this host's v1 generator),
TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
contract, and the archive invariants.
- homepage card + docs updated, and they were stale twice over: they still
described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.
Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
295 lines
12 KiB
Bash
Executable File
295 lines
12 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Run only the Bats tests affected by the staged paths.
|
|
#
|
|
# Usage:
|
|
# scripts/test-changed.sh # inspect the staged commit
|
|
# scripts/test-changed.sh --list # print selected test files
|
|
# scripts/test-changed.sh --paths FILE ...
|
|
#
|
|
# --paths is intended for tests and local experimentation. The normal mode is
|
|
# deliberately staged-only so a pre-commit hook cannot accidentally validate
|
|
# an unrelated dirty working-tree file.
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
LIST_ONLY=0
|
|
USE_PATHS=0
|
|
declare -a CHANGED=()
|
|
|
|
die() {
|
|
printf 'ERROR: %s\n' "$*" >&2
|
|
exit 2
|
|
}
|
|
|
|
while [ "$#" -gt 0 ]; do
|
|
case "$1" in
|
|
--list)
|
|
LIST_ONLY=1
|
|
shift
|
|
;;
|
|
--paths)
|
|
USE_PATHS=1
|
|
shift
|
|
while [ "$#" -gt 0 ] && [[ "$1" != --* ]]; do
|
|
CHANGED+=("$1")
|
|
shift
|
|
done
|
|
;;
|
|
--)
|
|
shift
|
|
CHANGED+=("$@")
|
|
break
|
|
;;
|
|
*)
|
|
die "unknown option '$1' (use --list or --paths FILE ...)"
|
|
;;
|
|
esac
|
|
done
|
|
|
|
cd "$REPO_ROOT"
|
|
if [ "$USE_PATHS" -eq 0 ]; then
|
|
while IFS= read -r -d '' path; do
|
|
CHANGED+=("$path")
|
|
done < <(git diff --cached --name-only -z --diff-filter=ACMRTUXB)
|
|
fi
|
|
|
|
[ "${#CHANGED[@]}" -gt 0 ] || {
|
|
printf 'No staged paths; no focused tests to run.\n'
|
|
exit 0
|
|
}
|
|
|
|
declare -a TESTS=()
|
|
add_test() {
|
|
local test="$1"
|
|
local existing
|
|
for existing in "${TESTS[@]:-}"; do
|
|
[ "$existing" != "$test" ] || return 0
|
|
done
|
|
TESTS+=("$test")
|
|
}
|
|
|
|
for path in "${CHANGED[@]}"; do
|
|
case "$path" in
|
|
tests/*.bats)
|
|
add_test "$path"
|
|
;;
|
|
tests/helpers.bash|tests/run.sh)
|
|
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
|
;;
|
|
.githooks/*)
|
|
add_test tests/changed_tests.bats
|
|
;;
|
|
dankserver/compose.yaml|dankclaw/compose.yaml)
|
|
add_test tests/compose_validity.bats
|
|
add_test tests/env_consistency.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankserver/Caddyfile|dankclaw/Caddyfile)
|
|
add_test tests/caddyfile.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
tests/e2e/*)
|
|
add_test tests/e2e_scaffold.bats
|
|
;;
|
|
dankserver/.env.example|dankclaw/.env.example)
|
|
add_test tests/env_consistency.bats
|
|
;;
|
|
shared/homepage/*)
|
|
add_test tests/homepage_caddy.bats
|
|
add_test tests/homepage_config.bats
|
|
add_test tests/homepage_deploy.bats
|
|
add_test tests/homepage_pages.bats
|
|
add_test tests/homepage_checker.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
shared/searxng/*)
|
|
add_test tests/searxng_limiter.bats
|
|
add_test tests/searxng_quadlet.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# The five observability containers left compose for Quadlet units on
|
|
# 2026-09-11. Their seam pins boot persistence and the loopback-only
|
|
# publish posture that keeps them off every host interface. This case
|
|
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
# would otherwise swallow it into unrelated seams.
|
|
dankclaw/observability/*)
|
|
add_test tests/observability_quadlet.bats
|
|
add_test tests/caddyfile.bats
|
|
add_test tests/deployment_scoping.bats
|
|
add_test tests/docs_completeness.bats
|
|
;;
|
|
# Caddy left compose.yaml on 2026-08-26 for a quadlet; caddyfile.bats
|
|
# now asserts against the quadlet itself. Route every change under
|
|
# dankclaw/caddy/ to both seams (this block must precede the
|
|
# dankclaw/*.container glob, which would otherwise swallow the unit).
|
|
dankclaw/caddy/*)
|
|
add_test tests/caddy_quadlet.bats
|
|
add_test tests/caddyfile.bats
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankclaw/*.container|dankclaw/*.network)
|
|
add_test tests/open_webui_quadlet.bats
|
|
add_test tests/pihole.bats
|
|
add_test tests/searxng_quadlet.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# Anubis is a quadlet + Caddyfile pair: the chain (public listener ->
|
|
# anubis:8923 -> internal :8016 listener) spans both, so any change
|
|
# under the service dir must re-run the caddyfile seam too. This case
|
|
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
# would otherwise swallow it into unrelated seams.
|
|
dankclaw/anubis/*)
|
|
add_test tests/anubis.bats
|
|
add_test tests/caddyfile.bats
|
|
add_test tests/homepage_caddy.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# frank-large is a rootless quadlet again since 2026-09-16 (the
|
|
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
|
# podman posture, so it gets its own seam rather than the generic
|
|
# fallback below.
|
|
dankclaw/frank-large/*)
|
|
add_test tests/frank_large_unit.bats
|
|
add_test tests/deployment_scoping.bats
|
|
add_test tests/docs_completeness.bats
|
|
;;
|
|
# Quadlets live under dankclaw/<svc>/quadlet/ (reorganized out of the
|
|
# top-level dankclaw/*.container globs above). Map the whole subtree so
|
|
# changes to the pihole, open-webui, searxng, and gitea quadlets hit
|
|
# their seam tests. The pihole-specific block must come first: shell
|
|
# case matches the first pattern, and dankclaw/pihole/quadlet/*.container
|
|
# also satisfies the dankclaw/*/quadlet/*.container fallback.
|
|
dankclaw/pihole/*)
|
|
add_test tests/pihole.bats
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankclaw/*/quadlet/*.container|dankclaw/*/quadlet/*.network)
|
|
add_test tests/open_webui_quadlet.bats
|
|
add_test tests/pihole.bats
|
|
add_test tests/searxng_quadlet.bats
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
|
add_test tests/deployment_flow.bats
|
|
add_test tests/deployment_scoping.bats
|
|
add_test tests/scripts_syntax.bats
|
|
;;
|
|
scripts/mount-entropy.sh)
|
|
add_test tests/mount_entropy.bats
|
|
;;
|
|
# frank-medium is a rootless quadlet again since the 2026-09-20
|
|
# tensor-parallel cutover: the TP=2 radiance unit, its entrypoint and
|
|
# its healthcheck all live under dankserver/frank-medium/, and the
|
|
# container mounts the latter two straight out of the checkout. The
|
|
# llama.cpp trio that held this endpoint before that date is archive
|
|
# material in scripts/.AGENTTRASH/ and maps to nothing.
|
|
dankserver/frank-medium/*)
|
|
add_test tests/frank_medium_unit.bats
|
|
add_test tests/deployment_scoping.bats
|
|
add_test tests/docs_completeness.bats
|
|
;;
|
|
# frank-small is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
|
# binary, so no generated quadlet is involved. Its env, unit and serve
|
|
# script all live in scripts/, which is why this case has to precede
|
|
# the generic scripts/*.service and scripts/* fallbacks below.
|
|
scripts/frank-small.*|scripts/serve-frank-small.sh)
|
|
add_test tests/frank_small_unit.bats
|
|
add_test tests/llm_units.bats
|
|
add_test tests/deployment_scoping.bats
|
|
add_test tests/scripts_syntax.bats
|
|
;;
|
|
scripts/*.service)
|
|
add_test tests/llm_units.bats
|
|
add_test tests/scripts_syntax.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
scripts/*)
|
|
add_test tests/scripts_syntax.bats
|
|
;;
|
|
serverAppConfigs-docs/*)
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankclaw/hermes/*)
|
|
add_test tests/docs_completeness.bats
|
|
;;
|
|
README.md|AGENTS.md)
|
|
# Project guidance changes can affect the documented deployment
|
|
# contract; validate the docs vault and scoped deployment seams.
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankclaw/comfyui/*)
|
|
add_test tests/docs_completeness.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# loki and alloy images are built from this repo's Containerfiles; the
|
|
# loki one carries the healthcheck command (see its comment), so both
|
|
# seams read it.
|
|
dankclaw/loki/*|dankclaw/alloy/*)
|
|
add_test tests/observability_quadlet.bats
|
|
add_test tests/caddyfile.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
dankclaw/udev/*)
|
|
# Host-level udev rules are applied by manual sudo install + udevadm
|
|
# trigger, not dispatched as a service; nothing here is bats-testable.
|
|
;;
|
|
.AGENTTRASH/*)
|
|
# Trash moves are not testable; skip.
|
|
;;
|
|
.gitignore|.env.example)
|
|
# Ignore/ignore-list changes change nothing at runtime; not testable.
|
|
;;
|
|
old/scripts/*|old/scripts/**)
|
|
add_test tests/scripts_syntax.bats
|
|
;;
|
|
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
|
|
# copyparty, plex, tautulli): the seam checks boot-persistence +
|
|
# v1-generator contract; deploy-flow seams guard the deployer routing.
|
|
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
|
|
add_test tests/dankserver_quadlet.bats
|
|
add_test tests/deployment_flow.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# FrankBook-Pro units live inside the podman machine VM; the seam
|
|
# pins boot persistence, the gluetun netns invariant, and the
|
|
# plan-locked volume layout. Deployer changes route here too.
|
|
frankbook-pro/*)
|
|
add_test tests/frankbook_quadlet.bats
|
|
add_test tests/deployment_flow.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
# Seerr unit on dankclaw: boot persistence, loopback-only port
|
|
# posture, and the deployer routing that installs it.
|
|
dankclaw/seerr/*)
|
|
add_test tests/seerr_quadlet.bats
|
|
add_test tests/deployment_flow.bats
|
|
add_test tests/deployment_scoping.bats
|
|
;;
|
|
report/*)
|
|
# Frozen audit reports are point-in-time artifacts; not testable.
|
|
;;
|
|
*)
|
|
die "no focused-test mapping for '$path'; add one before committing"
|
|
;;
|
|
esac
|
|
done
|
|
|
|
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
|
# empty selection; that is a pass, not a bats usage error.
|
|
[ "${#TESTS[@]}" -gt 0 ] || exit 0
|
|
|
|
if [ "$LIST_ONLY" -eq 1 ]; then
|
|
printf '%s\n' "${TESTS[@]}"
|
|
exit 0
|
|
fi
|
|
|
|
command -v bats >/dev/null 2>&1 || die 'bats is not installed'
|
|
printf 'Running focused tests for %d changed path(s):\n' "${#CHANGED[@]}"
|
|
printf ' %s\n' "${TESTS[@]}"
|
|
bats --print-output-on-failure "${TESTS[@]}"
|