Files
serverAppConfigs/tests/deployment_scoping.bats
T
Frank c2116658f8 feat(frank-medium): cut the endpoint over to radiance TP=2
The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:

  arm I  llama.cpp layer split 32:33 + DFlash2, Q8_0  geo decode  45.7 t/s, TTFT 0.72 s, burst-4  71.7 t/s
  arm L  llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4  54.3 t/s
  arm R  radiance vLLM TP=2 + DFlash2-FP8, MXFP4      geo decode  95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s

Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.

  - un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
    it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
    The replica pair and its affinity proxy stay archived; one process owns
    both cards now.
  - retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
    bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
    rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
    stay: they are the tooling that rollback needs.
  - deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
    re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
    the drafter, the pinned patch clone and the patched libr4d; retires the
    llama root unit into the home trash instead of relinking it; then
    install_quadlet + wait_container_healthy 1800s (first start compiles).
    retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
    a/b/proxy trio only, since frank-medium is installed again.
  - planner + test-changed map dankserver/frank-medium/** to frank-medium
    and its seam; the scripts-trio mapping is gone with the files.
  - tests/frank_medium_unit.bats rewritten for the container contract: boot
    persistence of a GENERATED unit (no enable on this host's v1 generator),
    TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
    the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
    was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
    contract, and the archive invariants.
  - homepage card + docs updated, and they were stale twice over: they still
    described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.

Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
2026-09-20 18:15:27 -07:00

237 lines
9.7 KiB
Bash

#!/usr/bin/env bats
# Seam: service-scoped deployment planning.
# A repository commit can contain changes for several workloads while only a
# subset is safe to restart. The planner must keep inference isolated from
# unrelated service changes and treat docs-only changes as a no-op.
load helpers
PLANNER="$REPO_ROOT/scripts/deploy-plan.py"
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
@test "deployment planner maps SearXNG changes to SearXNG only" {
run python3 "$PLANNER" --target dankclaw --paths \
shared/searxng/settings.yml serverAppConfigs-docs/Dankclaw/SearXNG.md
[ "$status" -eq 0 ]
[ "$output" = "searxng" ]
}
@test "deployment planner maps large-model changes to the large model only" {
run python3 "$PLANNER" --target dankclaw --paths \
scripts/frank-large.env
[ "$status" -eq 0 ]
[ "$output" = "frank-large" ]
}
@test "the retired NPU endpoint is absent from the deployment inventory" {
! grep -q '"frank-npu"' "$PLANNER"
! grep -q 'frank_npu\|frank-npu' "$REMOTE_DEPLOY"
}
@test "the retired ik chat template plans nothing, and its archive is still here" {
# scripts/chat-template.jinja existed only for the ik_llama.cpp runtime
# (retired 2026-09-16 in favour of halogen-flash-server again). The
# llama.cpp frank-small runtime serves the template the GGUF itself
# carries, which is the one with reasoning_effort, preserve_thinking and
# the tool-call grammar. Editing the archive must deploy nothing -- and the
# archive must still be present, so this cannot pass by the file having
# vanished.
[ -f "$REPO_ROOT/scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp" ]
run python3 "$PLANNER" --target dankclaw --paths scripts/chat-template.jinja
[ "$status" -eq 0 ]
[ -z "$output" ]
run python3 "$PLANNER" --target dankserver --paths scripts/chat-template.jinja
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "deployment planner maps the frank-medium container dir to frank-medium only" {
run python3 "$PLANNER" --target dankserver --paths \
dankserver/frank-medium/quadlet/frank-medium.container \
dankserver/frank-medium/radiance-entrypoint.sh \
dankserver/frank-medium/frank-medium-healthcheck.py
[ "$status" -eq 0 ]
[ "$output" = "frank-medium" ]
# The path is not the assertion: both retired shapes are still in the tree
# as archive, and a rule matching either would restart a service nobody
# runs. Assert the negatives explicitly, plus the llama trio's old paths,
# whose mapping was removed with the runtime.
run python3 "$PLANNER" --target dankserver --paths \
dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium-a.container
[ "$status" -eq 0 ]
[ -z "$output" ]
run python3 "$PLANNER" --target dankserver --paths \
scripts/.AGENTTRASH/20260920-llama-layersplit/frank-medium.env.llamacpp-layersplit \
scripts/frank-medium.env
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "the small endpoint is Dankclaw's again, and Dankserver's copy stays retired" {
run python3 "$PLANNER" --target dankclaw --paths \
scripts/frank-small.env scripts/frank-small.service scripts/serve-frank-small.sh
[ "$status" -eq 0 ]
[ "$output" = "frank-small" ]
# frank-small runs on Dankclaw on the RTX 3090; the Dankserver endpoint of
# that name was retired with the FreeToken runtime on 2026-09-15 and must
# not come back through a planner mapping.
run python3 "$PLANNER" --target dankserver --paths \
scripts/frank-small.env scripts/frank-small.service scripts/serve-frank-small.sh
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "documentation-only changes plan no runtime deployment" {
run python3 "$PLANNER" --target dankclaw --paths \
serverAppConfigs-docs/Architecture/Deployment\ Flow.md serverAppConfigs-docs/Dankclaw/SearXNG.md
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "deployment planner maps shared config to one service" {
run python3 "$PLANNER" --target dankclaw --paths \
shared/prometheus/prometheus.yml
[ "$status" -eq 0 ]
[ "$output" = "prometheus" ]
}
@test "deployment planner maps shared observability configs independently" {
run python3 "$PLANNER" --target dankclaw --paths shared/loki/loki-config.yaml
[ "$status" -eq 0 ]
[ "$output" = "loki" ]
run python3 "$PLANNER" --target dankclaw --paths shared/alloy/config.alloy
[ "$status" -eq 0 ]
[ "$output" = "alloy" ]
}
@test "deployment planner maps service-owned non-compose files" {
run python3 "$PLANNER" --target dankclaw --paths dankclaw/hermes/config.yaml.example
[ "$status" -eq 0 ]
[ "$output" = "hermes" ]
}
@test "homepage is owned by Dankclaw's consolidated proxy" {
run python3 "$PLANNER" --target dankserver --paths shared/homepage/index.html
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "compose planner narrows a compose edit to the changed service" {
local base head
base=$(git -C "$REPO_ROOT" rev-parse HEAD)
head=$(git -C "$REPO_ROOT" rev-parse HEAD)
run python3 "$PLANNER" --target dankclaw --repo "$REPO_ROOT" \
--base "$base" --head "$head" --paths dankclaw/compose.yaml
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "network-coupled SearXNG config changes include the required peers" {
run python3 "$PLANNER" --target dankclaw --paths \
dankclaw/searxng/quadlet/gluetun.container
[ "$status" -eq 0 ]
[ "$output" = $'gluetun\nsearxng\nmarreta' ]
}
@test "retiring the standalone Gitea proxy deploys the canonical Caddy" {
run python3 "$PLANNER" --target dankclaw --paths \
dankclaw/gitea/quadlet/gitea-caddy.container
[ "$status" -eq 0 ]
[ "$output" = "caddy" ]
}
@test "Gitea network changes exclude the retired standalone proxy" {
run python3 "$PLANNER" --target dankclaw --paths \
dankclaw/gitea/quadlet/gitea.network
[ "$status" -eq 0 ]
[ "$output" = $'gitea-postgres\ngitea-server' ]
}
@test "frankbook-pro gluetun changes carry the whole netns family; arr changes stay scoped" {
run python3 "$PLANNER" --target frankbook-pro --paths \
frankbook-pro/gluetun/quadlet/gluetun.container
[ "$status" -eq 0 ]
[ "$output" = $'gluetun\nprowlarr\nqbittorrent\nradarr\nsonarr\nlidarr\nreadarr' ]
run python3 "$PLANNER" --target frankbook-pro --paths \
frankbook-pro/sonarr/quadlet/sonarr.container
[ "$status" -eq 0 ]
[ "$output" = "sonarr" ]
}
@test "remote Caddy deployment no longer blocks on the retired Gitea proxy" {
! grep -q 'refusing to replace it with caddy' "$REMOTE_DEPLOY"
! grep -q 'podman container exists gitea-caddy &&' "$REMOTE_DEPLOY"
}
@test "remote Caddy deployment retires the installed Gitea proxy" {
grep -q 'retire_legacy_gitea_caddy' "$REMOTE_DEPLOY"
grep -q 'systemctl --user stop gitea-caddy.service' "$REMOTE_DEPLOY"
grep -q 'systemctl --user disable gitea-caddy.service' "$REMOTE_DEPLOY"
grep -q 'gitea-caddy.container' "$REMOTE_DEPLOY"
grep -q 'HOME/.AGENTTRASH' "$REMOTE_DEPLOY"
grep -q 'stage_legacy_gitea_caddy_state' "$REMOTE_DEPLOY"
}
@test "remote deployment inventory no longer accepts gitea-caddy as a service" {
! grep -qE 'dankclaw:gitea-caddy|gitea-caddy\) deploy' "$REPO_ROOT/scripts/deploy.sh" "$REPO_ROOT/scripts/deploy-remote.sh"
! grep -q '"gitea-caddy"' "$REPO_ROOT/scripts/deploy-plan.py"
}
@test "remote deployer carries per-service deployment state" {
[ -x "$REMOTE_DEPLOY" ]
grep -q 'deploy-state' "$REMOTE_DEPLOY"
grep -q 'STATE\[' "$REMOTE_DEPLOY"
grep -q 'mark_status.*pending\|status=pending' "$REMOTE_DEPLOY"
grep -q 'mark_status.*deployed\|status=deployed' "$REMOTE_DEPLOY"
}
@test "remote deployer routes inference restarts through selected handlers" {
# The runner may restart inference only from the service-dispatch path. It
# must not contain a host-wide restart loop.
grep -q 'deploy_llm_large' "$REMOTE_DEPLOY"
grep -q 'deploy_llm_medium' "$REMOTE_DEPLOY"
grep -q 'deploy_service' "$REMOTE_DEPLOY"
! grep -qE 'for .*frank-(small|large|npu)-?' "$REMOTE_DEPLOY"
}
@test "remote runner rejects services belonging to the other host" {
run env DEPLOY_STATE_DIR="$(mktemp -d)" REMOTE_DIR="$REPO_ROOT" \
bash "$REMOTE_DEPLOY" dankserver "$(git rev-parse HEAD)" frank-default-large
[ "$status" -eq 2 ]
}
@test "remote runner recognizes a pending service at the same revision" {
# Contract superseded 2026-09-17: recognition used to mean "retry and
# hard-fail at the dispatcher" (exit 2), which let a retired service's
# stale pending entry (frank-default-sightsound) block every deploy
# before the e2e gate. Recognition now means: name the entry, prune it
# to the trash, and keep the run going.
local state_dir="$(mktemp -d)"
printf 'revision=%s\nstatus=pending\n' "$(git rev-parse HEAD)" > "$state_dir/unknown-service"
run env DEPLOY_STATE_DIR="$state_dir" REMOTE_DIR="$REPO_ROOT" HOME="$(mktemp -d)" \
bash "$REMOTE_DEPLOY" dankclaw "$(git rev-parse HEAD)" --pending
[ "$status" -eq 0 ]
[[ "$output" == *"unknown-service"* ]]
[ ! -e "$state_dir/unknown-service" ]
}
@test "top-level deployer delegates to the service-scoped remote runner" {
grep -q 'deploy-remote.sh' "$REPO_ROOT/scripts/deploy.sh"
grep -qE 'SERVICE|service' "$REPO_ROOT/scripts/deploy.sh"
grep -q 'OLD_REVISION' "$REPO_ROOT/scripts/deploy.sh"
grep -q 'OLD_REVISION\|old checkout revision' "$REPO_ROOT/scripts/deploy.sh"
}
@test "legacy service deploy wrappers delegate to the scoped deployer" {
for script in deploy-searxng-quadlet.sh deploy-pihole-quadlet.sh deploy-open-webui-quadlet.sh; do
grep -q 'deploy.sh' "$REPO_ROOT/scripts/$script" || {
echo "$script does not delegate to deploy.sh"
return 1
}
done
}