dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

retire frank-medium: tests, planning and observability ground on frank-primary

Browse Source 
The frank-medium retirement tail the 2026-09-26 lucebox cutover left
uncommitted: the radiance quadlet tree, its swap/fetch tooling and its
seam test are deleted (archive copies live in dankserver/.AGENTTRASH/
20260926-frank-medium-radiance/ and scripts/.AGENTTRASH/20260926-
frank-medium-radiance/), the planner/allowlist/test-changed mappings
drop the service, and frank-large's chat template moves to scripts/
(its old home was the retired frank-medium dir).

Finished here:
- tests/frank_primary_unit.bats: the lucebox quadlet seam (boot
  persistence, host-network bind, HIP_VISIBLE_DEVICES=0,1, HIP-reserved
  env-name guard, checkout-mounted entrypoint/healthcheck, first-start
  patience, deployer payload/port gates).
- test-changed.sh: explicit .AGENTTRASH/ case — retirement commits move
  files to trash, and the wrapper refused those additions with "no
  focused-test mapping".
- observability: the vllm: namespace left with the radiance runtime.
  Rules keep only llamacpp:* series (prefix-cache reuse, spec-decode,
  queueing, endpoint-down); the dashboard drops the vLLM KV/latency
  panels; prometheus.yml documents that frank-primary (lucebox) has no
  /metrics and therefore no scrape job. llm-fleet.json was left
  corrupted mid-edit (missing opening brace) and is rewritten from HEAD.

Suite: 339 green (was 7 red).
This commit is contained in:
Frank
2026-09-26 19:25:46 -07:00
parent 32d264ccda
commit d7bf9f119a
22 changed files with 243 additions and 2001 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+7 -162
View File
@@ -95,7 +95,7 @@ service_is_allowed() {
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
dankserver:frank-medium|dankserver:frank-primary|dankserver:node-exporter|\
dankserver:frank-primary|dankserver:node-exporter|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr)
@@ -1178,159 +1178,6 @@ deploy_llm_small() {
wait_for_health frank-small http://127.0.0.1:7771/health 900
}
# deploy_llm_medium — frank-medium is the vllm-radiance stack as a
# rootless Quadlet since the 2026-09-20 cutover: one tensor-parallel (TP=2)
# engine across both R9700s, R4D one-shot all-reduce, DFlash2-FP8 drafter.
# The entrypoint is quantization-aware (config.json quant_method): the plain
# FP8 path, the MXFP4 patch-and-serve path, and the ParoQuant MXFP6 stack
# (PR #51's launch6_p kernel). It replaced mainline llama.cpp's layer split,
# which held the endpoint since 2026-09-16; the conversation battery measured
# radiance at 2.1x decode and a third of the time-to-first-token with 4x the
# concurrent throughput (scripts/bench-conversation.py, 2026-09-20).
#
# WHY THE PAIR IS BACK AND THE TRIO IS NOT: the 2026-09-16 cutover retired this
# unit because TP=2 could not dispatch a collective on the second card (B650
# chipset bridges, AtomicOpsCap: Routing-), and the two-replica + routing-proxy
# trio was the workaround. The board swap cleared the atomics wall, so one
# TP=2 process is both possible and the fastest shape; the replicas and their
# affinity proxy stay retired.
#
# HOST ASSETS. Nothing here installs the image's payloads: the checkpoints, the
# pinned radiance patch clone, the patched libr4d build and the Triton compile
# cache live outside the checkout (see the unit file's mount comments). The
# gates below refuse a deploy that would start a container guaranteed to die.
LLM_MEDIUM_IMAGE="${LLM_MEDIUM_IMAGE:-ggz14/vllm-radiance-mxfp4:0.13.0-e060895}"
LLM_MEDIUM_IMAGE_LOCAL="${LLM_MEDIUM_IMAGE_LOCAL:-localhost/radiance-vllm:0.13.0-e060895}"
LLM_MEDIUM_SNAP="${LLM_MEDIUM_SNAP:-/home/dank/models/Qwen3.8-27B-PARO-MXFP6}"
LLM_MEDIUM_DRAFTER="${LLM_MEDIUM_DRAFTER:-/home/dank/models/Qwen3.8-27B-DFlash2-FP8}"
LLM_MEDIUM_PATCHES="${LLM_MEDIUM_PATCHES:-/home/dank/GitPackages/radiance-vllm-mxfp4}"
LLM_MEDIUM_R4D="${LLM_MEDIUM_R4D:-/home/dank/.cache/radiance-libr4d/b9e42ab-rx6}"
LLM_MEDIUM_CACHE="${LLM_MEDIUM_CACHE:-/home/dank/.radiance-cache-mxfp6-013-pq-nqft-fp8s-gnq}"
# The three replica/proxy user units from the 2026-09-16 era are retired HERE,
# never left to rot: they must not exist alongside the TP=2 unit (the proxy
# published 7770 on the host network). Never `rm` — unit files move into the
# home trash; the images stay in local storage. `frank-medium` itself is NOT in
# the list: it is the unit this handler installs.
retire_vllm_replica_quadlets() {
local trash="$HOME/.AGENTTRASH/20260916-vllm-radiance-retire" unit
systemctl --user stop frank-medium-a.service frank-medium-b.service \
frank-medium-proxy.service 2>/dev/null || true
for unit in frank-medium-a frank-medium-b frank-medium-proxy; do
if [[ -e "$quadlet_unit_dir/$unit.container" ]]; then
mkdir -p "$trash"
mv -- "$quadlet_unit_dir/$unit.container" "$trash/$unit.container"
echo "retired quadlet $unit.container -> $trash"
fi
done
systemctl --user daemon-reload
}
# The llama.cpp root unit is retired HERE, into the home trash, never `rm`:
# /etc/systemd/system/frank-medium.service is a relink_unit symlink into this
# checkout, and the checkout pull that precedes this handler already moved the
# repo file away, leaving the symlink dangling. Stop it while systemd still has
# it loaded (an active unit stays loaded without its file), then take both the
# symlink and its [Install] alias. Nothing can start it at boot afterwards.
retire_llama_medium_unit() {
local trash="$HOME/.AGENTTRASH/20260920-llama-layersplit-frank-medium"
run_privileged systemctl stop frank-medium.service 2>/dev/null || true
if [[ -L /etc/systemd/system/frank-medium.service ||
-L /etc/systemd/system/multi-user.target.wants/frank-medium.service ]]; then
mkdir -p "$trash"
[[ -L /etc/systemd/system/frank-medium.service ]] &&
run_privileged mv -- /etc/systemd/system/frank-medium.service \
"$trash/etc-frank-medium.service.symlink"
[[ -L /etc/systemd/system/multi-user.target.wants/frank-medium.service ]] &&
run_privileged mv -- /etc/systemd/system/multi-user.target.wants/frank-medium.service \
"$trash/etc-frank-medium.wants.symlink"
echo "retired llama root unit frank-medium.service -> $trash"
fi
run_privileged systemctl daemon-reload
}
deploy_llm_medium() {
# Refuse rather than restart-loop. A container whose image or payload is
# missing never becomes healthy, and a green deploy would leave the endpoint
# dark with nothing in the deploy log to say why.
local missing=0
# Model-aware gates: read the serving shape from the quadlet (the source
# of truth), so a model swap is just "edit the quadlet + deploy". The
# container /models mount maps to /home/dank/models on the host. The
# drafter gate applies only to the dflash path; the patched libr4d gate
# applies to the mxfp4/quark/paroquant_mxfp6 path (the plain FP8 path uses
# the image's built-in r4d.so and skips the MXFP4 patches).
local quadlet_file="dankserver/frank-medium/quadlet/frank-medium.container"
local snap_c snap spec_method quant
snap_c="$(sed -nE 's/^Environment=SNAP=//p' "$quadlet_file" | head -1)"
spec_method="$(sed -nE 's/^Environment=SPEC_METHOD=//p' "$quadlet_file" | head -1)"
snap="/home/dank/models${snap_c#/models}"
quant="$(python3 -c 'import json,sys;print(json.load(open(sys.argv[1])).get("quantization_config",{}).get("quant_method",""))' "$snap/config.json" 2>/dev/null || true)"
[[ -n "$snap_c" ]] || { echo "ERROR: cannot read SNAP from $quadlet_file" >&2; return 1; }
if ! podman image exists "$LLM_MEDIUM_IMAGE_LOCAL"; then
if podman image exists "$LLM_MEDIUM_IMAGE"; then
podman tag "$LLM_MEDIUM_IMAGE" "$LLM_MEDIUM_IMAGE_LOCAL"
else
echo "ERROR: radiance image absent ($LLM_MEDIUM_IMAGE / $LLM_MEDIUM_IMAGE_LOCAL)" >&2
echo "frank-medium: podman pull $LLM_MEDIUM_IMAGE && podman tag $LLM_MEDIUM_IMAGE $LLM_MEDIUM_IMAGE_LOCAL" >&2
missing=1
fi
fi
if [[ ! -f "$snap/config.json" ]]; then
echo "ERROR: model checkpoint absent at $snap" >&2
missing=1
fi
if [[ "$spec_method" == "dflash" && ! -f "$LLM_MEDIUM_DRAFTER/config.json" ]]; then
echo "ERROR: DFlash2-FP8 drafter absent at $LLM_MEDIUM_DRAFTER" >&2
missing=1
fi
# The patches clone is mounted and its scripts run at EVERY container start
# on the MXFP4 path; the chat template is the fleet's reasoning-format
# contract on both paths.
if [[ ! -f "$LLM_MEDIUM_PATCHES/qwen-fixed-v22.3.jinja" ]]; then
echo "ERROR: pinned patch clone / chat template absent at $LLM_MEDIUM_PATCHES" >&2
echo "frank-medium: git clone https://github.com/ggz14/radiance-vllm-mxfp4 $LLM_MEDIUM_PATCHES" >&2
missing=1
fi
# The image's shipped r4d.so predates the gated-delta-net overflow fix and
# NaNs the MXFP4 model, so the patched build is not optional there. The
# plain FP8 path uses the image's built-in r4d.so (no MXFP4 patches).
case "$quant" in
mxfp4|quark|paroquant_mxfp6)
if [[ ! -f "$LLM_MEDIUM_R4D/r4d.so" ]]; then
echo "ERROR: patched libr4d absent at $LLM_MEDIUM_R4D" >&2
missing=1
fi
;;
esac
[[ "$missing" -eq 0 ]] || return 1
mkdir -p "$LLM_MEDIUM_CACHE"
retire_vllm_replica_quadlets
retire_llama_medium_unit
# 7770 is a host-network bind: nothing new can be created until the old
# holder released it (the same gate as the 2026-09-16 cutover, where the
# retired proxy held the port and the new engine failed at create). Sweep a
# stray manual llama-server too — that is how a bench harness leaves the
# port busy. `pkill -x` matches the process name exactly and can never
# match the deploying shell; never `pkill -f` here.
pkill -x llama-server 2>/dev/null || true
pkill -x VLLM::EngineCor 2>/dev/null || true
sleep 2
wait_port_free 7770 || return 1
install_quadlet frank-medium dankserver/frank-medium/quadlet frank-medium
# No enable: generated units cannot be enabled on this host (v1 quadlet
# generator); boot persistence is the unit's [Install] WantedBy=default.target.
systemctl --user restart frank-medium.service
# Triton/inductor compile on a cold cache runs minutes before the engine
# loads ~9.5 GiB per card; HealthStartPeriod in the unit is 1800s to match.
wait_container_healthy frank-medium 1800 || {
podman logs --tail 40 frank-medium 2>&1 || true
return 1
}
}
# deploy_llm_primary — frank-primary is the Lucebox DeepSeek V4 Quadlet
# (the 2026-09-26 cutover that replaces frank-medium on :7770). The model
# is a 377 GiB GGUF and the image is a local build, so the gates are:
@@ -1356,13 +1203,12 @@ deploy_llm_primary() {
fi
[[ "$missing" -eq 0 ]] || return 1
# Cutover: frank-primary takes :7770 from frank-medium. Stop and mask
# the radiance unit (masking is mandatory: Restart=on-failure would
# restart it and re-take the port). Retiring the radiance unit (moving
# it to the home trash) happens after frank-primary is verified — this
# handler only releases the port.
systemctl --user stop frank-medium 2>/dev/null || true
systemctl --user mask frank-medium
# Cutover COMPLETED 2026-09-26: frank-primary took :7770 and the radiance
# unit is retired. The host-side unit, its mask and the repo files were all
# moved to trash by the parent session, so there is nothing left to stop or
# mask here — masking a name that no longer exists would only resurrect it
# in the unit search path. The port gate stays, because 7770 is a
# host-network bind: nothing new can be created until the holder released it.
# 7770 is a host-network bind: nothing new can be created until the old
# holder released it (the same gate as the 2026-09-16 cutover).
sleep 2
@@ -1566,7 +1412,6 @@ deploy_service() {
marreta) deploy_marreta ;;
frank-large) deploy_llm_large ;;
frank-small) deploy_llm_small ;;
frank-medium) deploy_llm_medium ;;
frank-primary) deploy_llm_primary ;;
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;